https://github.com/facebookresearch/pythia
- HEAD
- refs/heads/0.1
- refs/heads/additional_validate
- refs/heads/airstore
- refs/heads/audio_video_encoders
- refs/heads/config_mmft_doc
- refs/heads/debug_ln_masked_lm
- refs/heads/dependabot/npm_and_yarn/website/ansi-regex-4.1.1
- refs/heads/dependabot/npm_and_yarn/website/http-cache-semantics-4.1.1
- refs/heads/dependabot/npm_and_yarn/website/terser-5.16.2
- refs/heads/dependabot/npm_and_yarn/website/webpack-5.76.1
- refs/heads/es
- refs/heads/fix
- refs/heads/fix_base_model
- refs/heads/fix_ci
- refs/heads/fix_circleci_tests
- refs/heads/fix_lint
- refs/heads/fixblack
- refs/heads/fsdp
- refs/heads/fsdp_asg
- refs/heads/fsdp_fairseq
- refs/heads/fsdp_support
- refs/heads/gh-pages
- refs/heads/gh/ebsmothers/1/base
- refs/heads/gh/ryan-qiyu-jiang/30/base
- refs/heads/gh/ryan-qiyu-jiang/30/head
- refs/heads/gh/ryan-qiyu-jiang/30/orig
- refs/heads/gh/ryan-qiyu-jiang/32/base
- refs/heads/gh/ryan-qiyu-jiang/32/head
- refs/heads/gh/ryan-qiyu-jiang/32/orig
- refs/heads/gh/ryan-qiyu-jiang/39/base
- refs/heads/gh/ryan-qiyu-jiang/39/head
- refs/heads/gh/ryan-qiyu-jiang/39/orig
- refs/heads/gh/ryan-qiyu-jiang/40/base
- refs/heads/gh/ryan-qiyu-jiang/40/head
- refs/heads/gh/ryan-qiyu-jiang/40/orig
- refs/heads/gh/ryan-qiyu-jiang/42/base
- refs/heads/gh/ryan-qiyu-jiang/42/head
- refs/heads/gh/ryan-qiyu-jiang/42/orig
- refs/heads/gh/ytsheng/2/base
- refs/heads/gh/ytsheng/2/orig
- refs/heads/gh/ytsheng/21/base
- refs/heads/gh/ytsheng/21/orig
- refs/heads/gh/ytsheng/3/base
- refs/heads/gh/ytsheng/3/orig
- refs/heads/gh/ytsheng/4/base
- refs/heads/gh/ytsheng/5/base
- refs/heads/gh/ytsheng/6/base
- refs/heads/gh/ytsheng/7/base
- refs/heads/gh/ytsheng/8/base
- refs/heads/hydra
- refs/heads/itm
- refs/heads/main
- refs/heads/mask_key_mmft
- refs/heads/mm_alignamet
- refs/heads/mmf_interactive
- refs/heads/mmft_output_key
- refs/heads/multitask_training
- refs/heads/notebooks
- refs/heads/pl_upgrade
- refs/heads/project/cycle-consistency
- refs/heads/project/m4c
- refs/heads/pt_19
- refs/heads/skip_optimizer_update
- refs/heads/stable_mmft
- refs/heads/v0.4
- refs/heads/vilbert_multimodal
- refs/heads/vilbert_multitask
- refs/heads/vilbert_multitask_flickr30k
- refs/heads/vilbert_multitask_visual7w
- refs/heads/xla-checkpoint-fix
- refs/tags/v0.3
- refs/tags/v0.3.1
Take a new snapshot of a software origin
If the archived software origin currently browsed is not synchronized with its upstream version (for instance when new commits have been issued), you can explicitly request Software Heritage to take a new snapshot of it.
Use the form below to proceed. Once a request has been submitted and accepted, it will be processed as soon as possible. You can then check its processing state by visiting this dedicated page.Processing "take a new snapshot" request ...
Permalinks
To reference or cite the objects present in the Software Heritage archive, permalinks based on SoftWare Hash IDentifiers (SWHIDs) must be used.
Select below a type of object currently browsed in order to display its associated SWHID and permalink.
Revision | Author | Date | Message | Commit Date |
---|---|---|---|---|
584e26d | Vedanuj Goswami | 08 September 2020, 22:59:00 UTC | [chores] Upgrade to pytorch 1.6 and torchvision 0.7 (#546) Summary: - Upgrading to pytorch 1.6, which comes with new features required for upcoming torchscript changes - Optimizer keys are no longer non-deterministic in PT1.6, check pytorch [PR](https://github.com/pytorch/pytorch/pull/37347 ). Remove `skip_keys` check from checkpoint optimizer tests. - Fixed nucleus sampling tests - Update docs to reflect support for pytorch 1.6 - Update CI tests, CircleCI and Actions to use pytorch 1.6, torchvision 0.7 Test Plan : - Tested with visual bert and mmbt models on Hateful Memes - Running internally at FB (runs on pytorch master) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/546 Reviewed By: apsdehal Differential Revision: D23558059 Pulled By: vedanuj fbshipit-source-id: 3d82978276462a3f3dea564ac33581240e284d03 | 08 September 2020, 23:00:27 UTC |
9e83320 | steph-en-m | 08 September 2020, 18:57:05 UTC | [feat] Add descriptive error message for missing processor in registry (#548) Summary: This PR resolves https://github.com/facebookresearch/mmf/issues/455 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/548 Reviewed By: vedanuj Differential Revision: D23559767 Pulled By: apsdehal fbshipit-source-id: 59e4e4e5d7dd9eaf88edf81b6906d5d8768f83a2 | 08 September 2020, 18:58:48 UTC |
3b8e26b | Amanpreet Singh | 06 September 2020, 01:59:53 UTC | [fix] Update HM docs for resume_pretrained=False during evaluation (#545) Summary: - Update README and docs Pull Request resolved: https://github.com/facebookresearch/mmf/pull/545 Reviewed By: vedanuj Differential Revision: D23545636 Pulled By: apsdehal fbshipit-source-id: bd23195e39584725b447d7be8ccedd0281a41d76 | 06 September 2020, 02:00:48 UTC |
efe4023 | steph-en-m | 04 September 2020, 19:56:04 UTC | [fix] modify mmf_convert_hm to create a copy of a file (#532) Summary: This PR resolves https://github.com/facebookresearch/mmf/issues/528 to ensure `mmf_convert_hm` preserves the original file Pull Request resolved: https://github.com/facebookresearch/mmf/pull/532 Test Plan: - tested it on the hateful memes dataset on colab and the original file is preserved. Reviewed By: apsdehal Differential Revision: D23545590 Pulled By: vedanuj fbshipit-source-id: 136f6bd9a526d4ed91fc27b4420160fbe9ff02e0 | 04 September 2020, 19:57:07 UTC |
637a5bc | Sasha Sheng | 04 September 2020, 04:53:35 UTC | [feat] added split dataset feature (#470) Summary: * allow user to specify x% of dataset to split from train, example yaml shown below: ``` textvqa: zoo_requirements: - textvqa.defaults - textvqa.ocr_en split_train: val: 0.2 test: 0.1 seed: 42 features: train: - textvqa/defaults/features/open_images/detectron.lmdb,textvqa/ocr_en/features/ocr_en_frcn_features.lmdb processors: ``` * To test: `pytests test` * See logs, 24221 is 0.7 of 34602 which is the total size of the train after calculation, etc. ![Screen Shot 2020-08-10 at 11 47 12 PM](https://user-images.githubusercontent.com/1545487/89866133-e052f600-db63-11ea-80e0-4f2c6b1a7acc.png) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/470 Reviewed By: apsdehal Differential Revision: D23098846 Pulled By: ytsheng fbshipit-source-id: 9498ae0eda8001716f8d8cf77724904a4e88a495 | 04 September 2020, 04:54:47 UTC |
1e64628 | rizavelioglu | 03 September 2020, 19:43:59 UTC | [fix] Typo in pretrain train+val config of visual bert (#541) Summary: fix typo in the name of the features files Pull Request resolved: https://github.com/facebookresearch/mmf/pull/541 Reviewed By: vedanuj Differential Revision: D23504310 Pulled By: apsdehal fbshipit-source-id: 9e3055a5150c98b8535edf3389804f6ecf509357 | 03 September 2020, 19:45:50 UTC |
7a35e17 | Amanpreet Singh | 03 September 2020, 17:15:14 UTC | [fix] Answer processor issues in MaskedVQA2Dataset (#529) Summary: - Adds add_answer option to default config - Set it to default in the dataset - Add default answer processor config Pull Request resolved: https://github.com/facebookresearch/mmf/pull/529 Test Plan: Tested on full masked_vqa2 config under VisualBERT Reviewed By: vedanuj Differential Revision: D23470296 Pulled By: apsdehal fbshipit-source-id: 8b8aacea829f2c4de392ab906de571fe8ec8d93d | 03 September 2020, 17:17:03 UTC |
98964eb | Amanpreet Singh | 03 September 2020, 16:58:55 UTC | [fix] Add missing init files to new folders (#531) Summary: - Without this, these folders are not included when installed through pip Pull Request resolved: https://github.com/facebookresearch/mmf/pull/531 Test Plan: Tested in separate repo Reviewed By: vedanuj Differential Revision: D23485913 Pulled By: apsdehal fbshipit-source-id: cc49fd7763016aa58ac8aa6dc47cd2e8a40fdf39 | 03 September 2020, 17:00:05 UTC |
24dedb1 | Amanpreet Singh | 01 September 2020, 23:08:45 UTC | [chores] Update contributing guidelines, issue templates, installation (#524) Summary: - Contributing guidelines now mention pre commit, our style guidelines and general PR suggestions - Adds issue template for a catch all category and enforce template always - Adds Pull request template - Updates installation docs to mention conda environment, windows install, installing directly from github with pip, and contributing guidelines along with recommended, optional tags for better clarity Pull Request resolved: https://github.com/facebookresearch/mmf/pull/524 Test Plan: Tested locally by building the website Reviewed By: vedanuj Differential Revision: D23407113 Pulled By: apsdehal fbshipit-source-id: 2da0bfe91838b56c039c8a34bda94140dd5f7c62 | 01 September 2020, 23:10:03 UTC |
d6fc967 | Vedanuj Goswami | 01 September 2020, 05:43:16 UTC | [fix] Fix lxmert model bbox processing (#526) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/520 and use `transformer_bbox_processor` processor - `transformer_bbox_processor` will now work when bbox features already have area norm Pull Request resolved: https://github.com/facebookresearch/mmf/pull/526 Test Plan: Run lxmert model for pretraining and finetuning on vqa2 Reviewed By: apsdehal Differential Revision: D23414310 Pulled By: vedanuj fbshipit-source-id: 0fa1848448ba193e0924966ae840ea225bb718ca | 01 September 2020, 05:44:20 UTC |
bf148a8 | Grzegorz Jacenków | 29 August 2020, 17:31:04 UTC | [docs] Fix tutorial for the Hateful Memes Challenge (#523) Summary: * Updates the documentation with a working code example. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/523 Reviewed By: vedanuj Differential Revision: D23407535 Pulled By: apsdehal fbshipit-source-id: e431f1a1ef6ee0105101d28701cec9bbdf9881b0 | 29 August 2020, 17:32:11 UTC |
cb3ecd1 | Amanpreet Singh | 28 August 2020, 22:17:09 UTC | [fix] Remove all .cuda calls from mmf so models can run on CPU (#522) Summary: - VisualBERT and VILBERT Pull Request resolved: https://github.com/facebookresearch/mmf/pull/522 Test Plan: Tested locally with and without GPU Reviewed By: vedanuj Differential Revision: D23404310 Pulled By: apsdehal fbshipit-source-id: e1e774e86d3726efd31271de1fe565b449739f2b | 28 August 2020, 22:18:47 UTC |
57ed846 | Suzanna Sia | 28 August 2020, 18:07:04 UTC | [fix] PathManager in sweep script as part of Manifold Migration (#498) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/498 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/132 Adopting `PathManager` for various file io utilities as part of Manifold migration. The default checkpoint_dir is still gluster, because I'm assuming not everyone has a manifold directory. To use a manifold_dir, we would sent it as an argument `--checkpoints_dir <manifold_save_dir>` via cli. Reasons for all the if-else statements within `fblearner.py` for manifold vs gluster, * this diff actually slows down submitting jobs to flow (because of the PathManager dependency). * It also requires a custom built kernel with torch instead of a regular bento kernel to run the flow script. Hen Other changes * simplified the logic of `has_started` * better logging of run in progress and what to do * check for `config.yaml` instead of `train.log` to determine if run exists Reviewed By: apsdehal Differential Revision: D23236008 fbshipit-source-id: 0aff4684934c82e0a3cc6f55143d55e5a4b6e719 | 28 August 2020, 18:08:10 UTC |
6d98b6c | Vedanuj Goswami | 28 August 2020, 15:49:43 UTC | [fix] Fix grad accumulation logic for Iterable datasets (#521) Summary: PR https://github.com/facebookresearch/mmf/issues/367 breaks training with `IterableDataset` as `len(self.train_loader) == 0` since `__len__` is not implemented for every `IterableDataset`. This PR mitigates this regression and : - Sets `num_remaining_batches` to be equal to (number of updates remaining x update frequency) for `IterableDataset` case. For non `IterableDataset` the old behavior is maintained (`len(self.train_loader)`) - Modifies the logic to avoid the need for checking if iterator has finished - Fixes another instance where we used to check `len(self.train_loader)` for setting number of updates when `max_epochs` is specified. Now in case of `IterableDataset` it will fall back to using `max_updates`. Added warning. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/521 Test Plan: - Tested internally on FB datasets - Tested on Hateful Memes Reviewed By: apsdehal Differential Revision: D23395374 Pulled By: vedanuj fbshipit-source-id: 415358455d12bdcb1d7ba87dd304c203c6cd990e | 28 August 2020, 15:51:08 UTC |
3f313b6 | Amanpreet Singh | 28 August 2020, 00:27:50 UTC | [feat] MultiClassFromFile Processor along with Violations (#519) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/519 - Adds violations dataset - Add processor for MultiClassFromFile - Adds batch processor for MutliClassFeaturesWithText Reviewed By: vedanuj Differential Revision: D23273579 fbshipit-source-id: 6f474f312014d41823689de2f7736b3563ff2b3f | 28 August 2020, 00:28:49 UTC |
ec1aa0f | Vedanuj Goswami | 27 August 2020, 18:02:02 UTC | [docs] FB internal documentation (#518) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/518 Reviewed By: apsdehal Differential Revision: D23365885 fbshipit-source-id: ffbbe65ee6e8e9ae4dd05eb054cdf38df945f5e1 | 27 August 2020, 18:03:44 UTC |
f5d2a88 | Vedanuj Goswami | 27 August 2020, 17:54:20 UTC | [feat] Open image features with PathManager (#468) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/468 - Adds ability to read features from different URIs (Manifold, Everstore URIs) in addition to file paths. - Adds target for lmdb conversion/extraction Reviewed By: apsdehal Differential Revision: D23021466 fbshipit-source-id: 01cc4811730a02c98b57896e122e4a044f4546a7 | 27 August 2020, 17:55:55 UTC |
78c5bfa | Vedanuj Goswami | 27 August 2020, 03:57:34 UTC | [docs] Update BUTD documentation (#516) Summary: Update BUTD documentation - Move to website docs - Add more instructions about inference - Added table with updated best model Pull Request resolved: https://github.com/facebookresearch/mmf/pull/516 Test Plan: NA Reviewed By: apsdehal Differential Revision: D23351839 Pulled By: vedanuj fbshipit-source-id: 58a79b4bea6484f10e93425dc6032a4163082461 | 27 August 2020, 03:58:38 UTC |
cf12af6 | Vedanuj Goswami | 27 August 2020, 03:49:24 UTC | [fix] Freeze CircleCI black version (#517) Summary: Freeze black and flake8 versions in CircleCI to be in sync with the pre-commit hooks Pull Request resolved: https://github.com/facebookresearch/mmf/pull/517 Test Plan: Pass CircleCI tests Reviewed By: apsdehal Differential Revision: D23359588 Pulled By: vedanuj fbshipit-source-id: bdf7eb75b26ef26996bbb5ebb207841d099a04b6 | 27 August 2020, 03:50:37 UTC |
31f0cc1 | Vedanuj Goswami | 27 August 2020, 02:50:05 UTC | [feat] Generalized MM transformer (#422) Summary: Introduces a generalized version of multimodal transformer. - Extends `BaseTransformer` and builds a modular multimodal transformer model - Base transformer model can be any model from [transformers](https://github.com/huggingface/transformers) - Supports n number of modalities - Adds ability to add separate input, position and segment embeddings for each modality - text modalities share weight with base transformer model embeddings - position embeddings use separate embeddings for each modality - token type/segment embeddings are shared/mean of base transformer token/segment embeddings (if available) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/422 Test Plan: - Tested with HM, VQA2, OKVQA Reviewed By: apsdehal Differential Revision: D22636744 Pulled By: vedanuj fbshipit-source-id: 3fa05472f3486a62fb6d3a1ffcd689712af076b4 | 27 August 2020, 02:51:25 UTC |
2db5ad3 | Ronghang Hu | 27 August 2020, 02:44:52 UTC | [fix] load lr scheduler state dict during checkpoint loading (#513) Summary: the current implementation does not load lr_scheduler's states from checkpoints, which is a bug (causing last_epoch to be reset when loading from checkpoints). This commit fixes it by saving and loading lr_scheduler's states. It fall backs to setting lr_scheduler's last_epoch when lr_scheduler's states don't exist in checkpoint. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/513 Test Plan: tested with M4C checkpoint saving and loading. (CPU tests failed due to black formatting error in mmf/datasets/processors/bert_processors.py, unrelated to this PR.) Reviewed By: apsdehal Differential Revision: D23349944 Pulled By: ronghanghu fbshipit-source-id: 5c0802957804d3e169f30f0b944aa3d54ccf956a | 27 August 2020, 02:46:02 UTC |
aab972e | Eric Undersander | 27 August 2020, 02:39:13 UTC | [feature, fix] added support for update_frequency (v2) (#367) Summary: This PR adds support for config.training.update_frequency: ``` # Run update_frequency=K batches with batch_size=N, accumulate gradients, then do # one update (one optimizer step). The effect is a large effective batch size of # KxN (without incurring the memory overhead of setting batch_size to KxN). update_frequency: 1 ``` https://www.internalfb.com/intern/tasks/?t=66569918 This is a revised approach; see also earlier PR https://github.com/facebookresearch/mmf/pull/292. With this change, we have to be more clear with naming. Amanpreet and I agreed on this: - we have no particular term for a single forward/backward pass for a single batch - "update" == "iteration" == one optimizer step (will be more than one batch when update_frequency > 1) The update logic in `BaseTrainer._backward` has mostly been moved to `_start_update` and `_finish_update`. There's also a fix for `MultiDatasetLoader.__len__`. The existing bug is only present for the case that the train set size is not a multiple of per-worker batch size, which is probably rare in practice. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/367 Test Plan: I did ad-hoc testing of BaseTrainer.train() (`mmf_run config="projects/hateful_memes/configs/late_fusion/defaults.yaml" model="late_fusion" dataset=hateful_memes`) with print statements and varying `[batch_size, update_frequency, train set size]`, including end-of-epoch and completion of the train run. Reviewed By: vedanuj Differential Revision: D23335921 Pulled By: apsdehal fbshipit-source-id: 984ae2706e50679eb0be47e14d872a359de36fd6 | 27 August 2020, 02:41:45 UTC |
a5096b4 | John Knox | 26 August 2020, 15:40:54 UTC | [mmf] Use static docs plugin Summary: I noticed mmf doesn't have the static docs plugin enabled. This will make sure your browser address bar and page titles work as expected. Reviewed By: apsdehal Differential Revision: D23343272 fbshipit-source-id: 1de615c72968722da162bf443d386301cbe60063 | 26 August 2020, 15:41:55 UTC |
f700be8 | John Knox | 26 August 2020, 15:11:11 UTC | [staticdocs] Update plugin version for all consumers Summary: Updates the static docs plugin everywhere. Done by running `./update-all-users.sh 0.3.6` inside `fbsource/xplat/staticdocs/` Reviewed By: mweststrate Differential Revision: D23343132 fbshipit-source-id: 2deaec1f8a4ead43475d77f40b7bf35e6b25f994 | 26 August 2020, 15:12:17 UTC |
53ed195 | Amanpreet Singh | 25 August 2020, 23:07:18 UTC | [fix] Unimodal base in case of batch size 2 (#502) Summary: Current implementation uses `len` to check whether output was from bert encoder or not. Add extra check for `torch.is_tensor` so that image features case for batch size 2 safely works. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/502 Test Plan: Tested on batch size 2. Reviewed By: vedanuj Differential Revision: D23243300 Pulled By: apsdehal fbshipit-source-id: 120469dc4ced94dd8348285223213d4409a3b22b | 25 August 2020, 23:08:58 UTC |
298718e | Vedanuj Goswami | 24 August 2020, 22:54:13 UTC | [fix] Fix multilabel macro-f1 inheritance (#508) Summary: - Fixes MultiLabel Macro F1 metric. It should inherit from the `MultiLabelF1`. - Fixes an issue with a recent change in pytorch 1.7, where earlier argmax would return the index of ANY min or max, but now they will return the index of the FIRST min or max. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/508 Test Plan: Tests passed Reviewed By: apsdehal Differential Revision: D23283349 Pulled By: vedanuj fbshipit-source-id: e52dac18eed875767f251ef491329d1d5093bbf1 | 24 August 2020, 22:56:12 UTC |
ab8d3f5 | Vedanuj Goswami | 22 August 2020, 01:50:19 UTC | [feat] Add support for BS, NS metric calculation, update butd model etc. (#495) Summary: Fixes https://github.com/facebookresearch/mmf/issues/482 #489 Previously caption bleu4 metric in mmf could not be directly used with Beam Search(BS), Nucleus Sampling(NS). The process to follow was extract the predictions and then compute metrics separately. - Adds support for calculating metrics when using BS, NS - Uploaded new model to zoo trained with latest features. Old model when used with new features with mmf doesn't work. Issue https://github.com/facebookresearch/mmf/issues/489 - Modified `accumulate_tensor_fields` to throw a warning instead of KeyError when some metric field is not available. This is done to support use cases where different model output needs to be accumulated depending on train/val/test stages. - Fix zoo requirement for coco captioning - Fixes the issue of CUDA OOM when running inference with BS, NS on a GPU with less memory. Now only the captions(bs x cap_len) are used instead of scores (bs x cap_len x vocab_sz). Pull Request resolved: https://github.com/facebookresearch/mmf/pull/495 Test Plan: BLEU4 Score on val set : 36.04 ``` mmf_run config=projects/butd/configs/coco/beam_search.yaml run_type=val dataset=coco model=butd checkpoint.resume_zoo=butd ``` Reviewed By: apsdehal Differential Revision: D23214751 Pulled By: vedanuj fbshipit-source-id: 27f0cdc21cf09061d82a580440f4fa8ee13a1016 | 22 August 2020, 01:51:58 UTC |
95e89ee | Amanpreet Singh | 20 August 2020, 15:56:35 UTC | [feat] Support for more than 2 segments in MMBT (#460) Summary: Segment embedding other than first 2 dim are initialized from the average of first 2 embeddings This is required for the case where there are 3 segments such as post, ocr, and features. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/460 Reviewed By: vedanuj Differential Revision: D22970794 Pulled By: apsdehal fbshipit-source-id: 4d5ee86162079ae96d5387b0c08e96d73bb3bbe1 | 20 August 2020, 15:57:56 UTC |
697ca3b | Suzanna Sia | 20 August 2020, 04:17:32 UTC | [fix] Support for reading raw COCO images for VQA2 and COCO to support models like MMBT (#459) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/459 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/127 VQA2 and COCO dataset readers `dataset.py` previously only support reading in from feature file. After this PR, it should now be compatible with a model which reads in raw images. Should not affect any models which previously relied on `use_images: false`, `use_features: true` Changes: * Loads images in `{vqa2,coco}/{masked_,}dataset.py` if `self._use_features` flag is False * Included image transform in `faim/mmf/mmf/datasets/builders/vqa2/dataset.py:def init_processors(self)` * Enabled npy loading for internal `faim/mmf/mmf/datasets/databases/annotation_database.py:def _load_npy()` * Configs -- The main difference for configs is the inclusion of image_processor in this config file: `mmf/configs/datasets/vqa2/with_raw_images.yaml` and the dataset config `use_images` and `use_features` flags. ``` dataset_config: vqa2: use_images: true use_features: false images: train: - coco/defaults/images/train2014 val: - coco/defaults/images/val2014 test: - coco/defaults/images/test2015 ``` Reviewed By: apsdehal Differential Revision: D22979686 fbshipit-source-id: 3cac3d41329c098e0db1b50e4c554b72a194c120 | 20 August 2020, 04:18:37 UTC |
16b20ff | Amanpreet Singh | 20 August 2020, 01:52:06 UTC | [fix] Equal size proportional sampling and memory leak (#473) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/472 Here is the description that have been added to dataset loader on how sampling works after this change: Calculation of next batch is performed using following logic. Current chosen iterator is selected based on the dataset probabilities set in the `change_dataloader` function which is called every time `prepare_batch` is called. If we get the next batch from iterator without any StopIteration exception, we return it as it is. Otherwise, we have two cases: 1. In proportional sampling, since each dataset will have same number of epochs at any given time, we need to yield StopIteration exception when all iterators are finished. In turn, this will yield to `__iter__` all reignite all of the iterators. The code will not reach `__iter__` until unless all iterators are exhausted. 2. In the case of non-proportional sampling, epochs don't make sense. Think of a case of equal proportion sampling for dataset x and y where x is half the size of y. When x will complete its 2 epochs, y will have only 1 epoch completed. **So please don't use max_epochs or epoch based training in this case as it won't be honored**. If an iterator is finished, we just reignite it in this case and finished iterators variable isn't used. This means that this case will never reach the `__iter__` function ever again. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/473 Test Plan: Tests have been added to check size proportional sampling and equal sampling. Reviewed By: vedanuj Differential Revision: D23089370 Pulled By: apsdehal fbshipit-source-id: 06c09746624a32cb8bde5e52369d8891c34aa7e1 | 20 August 2020, 01:53:10 UTC |
56e585d | 在原佐为 | 19 August 2020, 18:20:11 UTC | [fix] package configs in sdist package (#488) Summary: add config in MANIFEST.in to package yaml in mmf/configs. see https://github.com/facebookresearch/mmf/issues/479 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/488 Reviewed By: vedanuj Differential Revision: D23202899 Pulled By: apsdehal fbshipit-source-id: 6544e20e458c103912cdc9abbc4cf86d0b71ff73 | 19 August 2020, 18:21:40 UTC |
6b94a8b | Antonio V Mendoza | 19 August 2020, 03:53:11 UTC | [feat] Adding LXMERT to list of available VL models for finetuning and pretraining (#339) Summary: Adding model LXMERT. To pretrain on VQA run: `mmf_run config=projects/lxmert/configs/vqa2/pretrain.yaml run_type=train_val dataset=masked_vqa2 model=lxmert ` Added config files for GQA, COCO, VQA2.0, Visual Genome pretraining and VQA2 finetuning. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/339 Reviewed By: apsdehal Differential Revision: D22858282 Pulled By: vedanuj fbshipit-source-id: 09c8c04458df88866bf95ed6ed466042eb969900 | 19 August 2020, 03:54:16 UTC |
962a7ea | Suzanna Sia | 19 August 2020, 00:43:08 UTC | [fix] More informative logging for loading checkpoints. (#491) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/491 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/131 Previously unclear at what stage the loaded checkpoint file resumes training and this may have some unintended consequences. Adding `num_updates`, `current_iteration`, and `current_epoch` to the logger. Reviewed By: vedanuj Differential Revision: D23199211 fbshipit-source-id: c224d169f2aad681e235a3b820b2b824b5021055 | 19 August 2020, 00:44:25 UTC |
da0b126 | Vedanuj Goswami | 19 August 2020, 00:12:58 UTC | [fix] Set proper max_features for movie_mcan model (#492) Summary: Fixes https://github.com/facebookresearch/mmf/issues/484 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/492 Reviewed By: apsdehal Differential Revision: D23202564 Pulled By: vedanuj fbshipit-source-id: 459804792cc8bfc674ec06b0fb28fa973343b571 | 19 August 2020, 00:14:17 UTC |
7b968a7 | Vedanuj Goswami | 18 August 2020, 20:15:09 UTC | [chores] Add targets for sweep scripts (#130) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/130 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/490 Sweep script changes Reviewed By: lichengunc Differential Revision: D23183455 fbshipit-source-id: fcd1ead78170b95f333ac3cd1419d12be916506b | 18 August 2020, 20:16:43 UTC |
1fa896f | Ning Zhang | 18 August 2020, 04:15:25 UTC | [fix] Typo in fusions for image feature (#129) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/129 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/487 Change image_features_0 to image_feature_0. All other image tokens are named as image_feature_0. Reviewed By: apsdehal Differential Revision: D23178741 fbshipit-source-id: ed26d445e4d3af5e7afa39f561cdaad14e80071f | 18 August 2020, 04:16:48 UTC |
9d76995 | Amanpreet Singh | 18 August 2020, 02:28:15 UTC | [fix] Remove id from required params (#486) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/485 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/486 Reviewed By: vedanuj Differential Revision: D23181522 Pulled By: apsdehal fbshipit-source-id: 7efb7db5a6868e3c8d943e74cfc8f2f551168145 | 18 August 2020, 02:29:19 UTC |
2811c1e | Amanpreet Singh | 14 August 2020, 21:45:45 UTC | [refactor] Mimic formatter settings same as fbcode (#474) Summary: - Removes seed-isort-config - Seeds all third_party once, in future authors should manually update this - Add MMF also to third party to be comparable with internal - Pin black and isort versions same as internal - Update all files Pull Request resolved: https://github.com/facebookresearch/mmf/pull/474 Reviewed By: vedanuj Differential Revision: D23092121 Pulled By: apsdehal fbshipit-source-id: 9b7420408dca7ab588dc94b5d886073dcdad89c9 | 14 August 2020, 21:47:50 UTC |
fda78ad | Vedanuj Goswami | 13 August 2020, 05:05:37 UTC | [feat] Padded grid features (#464) Summary: Fixes issue https://github.com/facebookresearch/mmf/issues/453 . - Add support for variable feature size for grid based features. - This will enable using features from https://github.com/facebookresearch/grid-feats-vqa repository Pull Request resolved: https://github.com/facebookresearch/mmf/pull/464 Test Plan: Tested Movie+Mcan model with variable sized features. Reviewed By: apsdehal Differential Revision: D23054193 Pulled By: vedanuj fbshipit-source-id: 8fd1a96671e6bca0d2763398056014f20a8da13c | 13 August 2020, 05:06:42 UTC |
21af61a | Vedanuj Goswami | 13 August 2020, 00:13:09 UTC | [mmf][feat] Allow option to override concat in multi sentence tokenizer (#471) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/471 Adding a config param for concatenating tokenizer output for multi sentence tokenizer. This is required for models which build separate embeddings for each sentence/modality in a sample. Reviewed By: apsdehal Differential Revision: D23070178 fbshipit-source-id: 2e3b3232c564d5eb46c304af6760348d98210966 | 13 August 2020, 00:14:36 UTC |
918092f | Yingxin Kang | 12 August 2020, 19:00:44 UTC | [feat] Install glove only when the processor is called (#469) Summary: Embeddings download happens in Vocab(..), which was previously called in __init__. We put it in __call__ instead to delay the download. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/469 Reviewed By: vedanuj, apsdehal Differential Revision: D23046828 Pulled By: Miiira fbshipit-source-id: 061d238ead0aa9dd3b10ff9cd0fe0abe7c74215a | 12 August 2020, 19:02:03 UTC |
6ac815d | Suzanna Sia | 12 August 2020, 08:39:45 UTC | [fix] Allow user to define 'text_b' for bert_tokenizer text processor. (#128) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/128 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/461 Previously `bert_tokenizer` text processor assumed there are no `text_b` tokens. We still preserve this default logic (tokens_b defaults to None), but additionally allow the user to specify `text_b` if desired. Reviewed By: apsdehal Differential Revision: D22988310 fbshipit-source-id: 4d918380a59a4112483c95ec5cbdfd3641240dcf | 12 August 2020, 08:40:52 UTC |
1164df9 | dependabot[bot] | 11 August 2020, 19:47:55 UTC | Bump prismjs from 1.20.0 to 1.21.0 in /website (#463) Summary: Bumps [prismjs](https://github.com/PrismJS/prism) from 1.20.0 to 1.21.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/PrismJS/prism/releases">prismjs's releases</a>.</em></p> <blockquote> <h2>v1.21.0</h2> <p>Release 1.21.0</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PrismJS/prism/blob/master/CHANGELOG.md">prismjs's changelog</a>.</em></p> <blockquote> <h2>1.21.0 (2020-08-06)</h2> <h3>New components</h3> <ul> <li><strong>.ignore</strong> & <strong>.gitignore</strong> & <strong>.hgignore</strong> & <strong>.npmignore</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2481">#2481</a>) <a href="https://github.com/PrismJS/prism/commit/3fcce6fe"><code>3fcce6fe</code></a></li> <li><strong>Agda</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2430">#2430</a>) <a href="https://github.com/PrismJS/prism/commit/3a127c7d"><code>3a127c7d</code></a></li> <li><strong>AL</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2300">#2300</a>) <a href="https://github.com/PrismJS/prism/commit/de21eb64"><code>de21eb64</code></a></li> <li><strong>Cypher</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2459">#2459</a>) <a href="https://github.com/PrismJS/prism/commit/398e2943"><code>398e2943</code></a></li> <li><strong>Dhall</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2473">#2473</a>) <a href="https://github.com/PrismJS/prism/commit/649e51e5"><code>649e51e5</code></a></li> <li><strong>EditorConfig</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2471">#2471</a>) <a href="https://github.com/PrismJS/prism/commit/ed8fff91"><code>ed8fff91</code></a></li> <li><strong>HLSL</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2318">#2318</a>) <a href="https://github.com/PrismJS/prism/commit/87a5c7ae"><code>87a5c7ae</code></a></li> <li><strong>JS stack trace</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2418">#2418</a>) <a href="https://github.com/PrismJS/prism/commit/ae0327b3"><code>ae0327b3</code></a></li> <li><strong>PeopleCode</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2302">#2302</a>) <a href="https://github.com/PrismJS/prism/commit/bd4d8165"><code>bd4d8165</code></a></li> <li><strong>PureBasic</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2369">#2369</a>) <a href="https://github.com/PrismJS/prism/commit/d0c1c70d"><code>d0c1c70d</code></a></li> <li><strong>Racket</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2315">#2315</a>) <a href="https://github.com/PrismJS/prism/commit/053016ef"><code>053016ef</code></a></li> <li><strong>Smali</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2419">#2419</a>) <a href="https://github.com/PrismJS/prism/commit/22eb5cad"><code>22eb5cad</code></a></li> <li><strong>Structured Text (IEC 61131-3)</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2311">#2311</a>) <a href="https://github.com/PrismJS/prism/commit/8704cdfb"><code>8704cdfb</code></a></li> <li><strong>UnrealScript</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2305">#2305</a>) <a href="https://github.com/PrismJS/prism/commit/1093ceb3"><code>1093ceb3</code></a></li> <li><strong>WarpScript</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2307">#2307</a>) <a href="https://github.com/PrismJS/prism/commit/cde5b0fa"><code>cde5b0fa</code></a></li> <li><strong>XML doc (.net)</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2340">#2340</a>) <a href="https://github.com/PrismJS/prism/commit/caec5e30"><code>caec5e30</code></a></li> <li><strong>YANG</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2467">#2467</a>) <a href="https://github.com/PrismJS/prism/commit/ed1df1e1"><code>ed1df1e1</code></a></li> </ul> <h3>Updated components</h3> <ul> <li>Markup & JSON: Added new aliases (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2390">#2390</a>) <a href="https://github.com/PrismJS/prism/commit/9782cfe6"><code>9782cfe6</code></a></li> <li>Fixed several cases of exponential backtracking (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2268">#2268</a>) <a href="https://github.com/PrismJS/prism/commit/7a554b5f"><code>7a554b5f</code></a></li> <li><strong>APL</strong> <ul> <li>Added <code>⍥</code> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2409">#2409</a>) <a href="https://github.com/PrismJS/prism/commit/0255cb6a"><code>0255cb6a</code></a></li> </ul> </li> <li><strong>AutoHotkey</strong> <ul> <li>Added missing <code>format</code> built-in (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2450">#2450</a>) <a href="https://github.com/PrismJS/prism/commit/7c66cfc4"><code>7c66cfc4</code></a></li> <li>Improved comments and other improvements (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2412">#2412</a>) <a href="https://github.com/PrismJS/prism/commit/ddf3cc62"><code>ddf3cc62</code></a></li> <li>Added missing definitions (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2400">#2400</a>) <a href="https://github.com/PrismJS/prism/commit/4fe03676"><code>4fe03676</code></a></li> </ul> </li> <li><strong>Bash</strong> <ul> <li>Added <code>composer</code> command (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2298">#2298</a>) <a href="https://github.com/PrismJS/prism/commit/044dd271"><code>044dd271</code></a></li> </ul> </li> <li><strong>Batch</strong> <ul> <li>Fix escaped double quote (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2485">#2485</a>) <a href="https://github.com/PrismJS/prism/commit/f0f8210c"><code>f0f8210c</code></a></li> </ul> </li> <li><strong>C</strong> <ul> <li>Improved macros and expressions (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2440">#2440</a>) <a href="https://github.com/PrismJS/prism/commit/8a72fa6f"><code>8a72fa6f</code></a></li> <li>Improved macros (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2320">#2320</a>) <a href="https://github.com/PrismJS/prism/commit/fdcf7ed2"><code>fdcf7ed2</code></a></li> </ul> </li> <li><strong>C#</strong> <ul> <li>Improved pattern matching (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2411">#2411</a>) <a href="https://github.com/PrismJS/prism/commit/7f341fc1"><code>7f341fc1</code></a></li> <li>Fixed adjacent string interpolations (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2402">#2402</a>) <a href="https://github.com/PrismJS/prism/commit/2a2e79ed"><code>2a2e79ed</code></a></li> </ul> </li> <li><strong>C++</strong> <ul> <li>Added support for default comparison operator (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2426">#2426</a>) <a href="https://github.com/PrismJS/prism/commit/8e9d161c"><code>8e9d161c</code></a></li> <li>Improved class name detection (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2348">#2348</a>) <a href="https://github.com/PrismJS/prism/commit/e3fe9040"><code>e3fe9040</code></a></li> <li>Fixed <code>enum class</code> class names (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2342">#2342</a>) <a href="https://github.com/PrismJS/prism/commit/30b4e254"><code>30b4e254</code></a></li> </ul> </li> <li><strong>Content-Security-Policy</strong> <ul> <li>Fixed directives (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2461">#2461</a>) <a href="https://github.com/PrismJS/prism/commit/537a9e80"><code>537a9e80</code></a></li> </ul> </li> <li><strong>CSS</strong> <ul> <li>Improved url and added keywords (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2432">#2432</a>) <a href="https://github.com/PrismJS/prism/commit/964de5a1"><code>964de5a1</code></a></li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PrismJS/prism/commit/187c8a607ee70c7914682870156faa31ed01f001"><code>187c8a6</code></a> 1.21.0</li> <li><a href="https://github.com/PrismJS/prism/commit/bf4f323391d546d4b2e1a388f05c512a27491e9c"><code>bf4f323</code></a> Changelog for v1.21.0 (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2507">#2507</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/8bba4880202ef6bd7a1e379fe9aebe69dd75f7be"><code>8bba488</code></a> Previewers: Fixed XSS (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2506">#2506</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/158caf52343e59a66c2351ff1d83648efe871e33"><code>158caf5</code></a> JSON: Greedy comments (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2479">#2479</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/f0f8210c1a9745c064d49bfb985544c654986b24"><code>f0f8210</code></a> Batch: Fix escaped double quote (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2485">#2485</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/649e51e56250a81dc0b0c4f5b3a4ea23e1c21834"><code>649e51e</code></a> Added support for Dhall (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2473">#2473</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/453079bf96746e0c44f90cb7cd90fcae9a5f94cc"><code>453079b</code></a> Line Numbers: Fixed class name on website</li> <li><a href="https://github.com/PrismJS/prism/commit/a0efa40bde420ac3923947be611ebe6b3f077dce"><code>a0efa40</code></a> Fixed Treeview page (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2484">#2484</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/78161d607fe7d493831ff19759aac951330134a1"><code>78161d6</code></a> VB: Added VBA alias (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2469">#2469</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/ed1df1e1208401a8b84330ecc39689fa37d0e9f6"><code>ed1df1e</code></a> Added support for YANG (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2467">#2467</a>)</li> <li>Additional commits viewable in <a href="https://github.com/PrismJS/prism/compare/v1.20.0...v1.21.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=prismjs&package-manager=npm_and_yarn&previous-version=1.20.0&new-version=1.21.0)](https://help.github.com/articles/configuring-automated-security-fixes) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/463 Reviewed By: apsdehal Differential Revision: D23031582 Pulled By: vedanuj fbshipit-source-id: 18eaa0b3ccdc135f261d195d0e195f2b5f0e929d | 11 August 2020, 19:50:04 UTC |
67390cb | Parth Suresh | 10 August 2020, 18:00:27 UTC | [feat] Added VQACPv2 Dataset Implementation (#448) Summary: Adds support for VQACPv2 dataset in MMF. Automatic downloads are only supported at image level. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/448 Test Plan: A config has been added to test on MMBT with Images. `mmf_run config=projects/mmbt/configs/vqacp_v2/with_images.yaml run_type=train_val dataset=vqacp_v2 model=mmbt training.batch_size=64 training.seed=1` Reviewed By: vedanuj Differential Revision: D22928390 Pulled By: apsdehal fbshipit-source-id: 7aaac7581b96ace47cbd99a8cbf7fbb852c0519b | 10 August 2020, 18:02:18 UTC |
080a330 | Amanpreet Singh | 07 August 2020, 06:42:46 UTC | [feat] Processor Pipeline for internal batch processors (#462) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/462 Decouples the pipeline for feature and text processor into separate functions and then allow it to be extensible. Users can inherit this processor and define and override some particular parts. This will allow easy extensibility in future. Also adds MultiSentenceBertTokenizer into the mix. Reviewed By: vedanuj Differential Revision: D22961076 fbshipit-source-id: f08f8e76fb049b3e97a321a525c1c0c00cf0ad15 | 07 August 2020, 06:44:11 UTC |
61f14ba | Amanpreet Singh | 06 August 2020, 23:48:07 UTC | [feat] Synchronize device for batch based on model (#456) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/456 Purpose of this diff is to add `to_device` method which is a helper utility method to move sample list to a proper device. In case the user is using custom dataset and forgot to move it to proper device, this changes will make sure that the batch is automatically moved to proper device at multiple stages of a training loop including making sure the batch model is receiving is on same device as the model's device. The checks are added at the following stages: - Before forward in training loop - Before forward in prediction loop - Inside BaseModel's forward This diff also remove the warning about prepare_batch in the multi dataset loader as it is not needed anymore. Reviewed By: vedanuj Differential Revision: D22955817 fbshipit-source-id: 5cd69cff518480f75dd8e9eda14aca31e575c48b | 06 August 2020, 23:49:30 UTC |
5b60950 | Amanpreet Singh | 06 August 2020, 20:53:01 UTC | [feat] Allow custom keys in metrics and some new metrics (#447) Summary: - Users can specify custom keys for the metrics, for example in case of recall@k you can specify recall@50 as key and then pass parameter k - Add binary_ap and recall@precisionK metrics - Fix MicroAP and MacroAP inheritance Pull Request resolved: https://github.com/facebookresearch/mmf/pull/447 Test Plan: Tests added and modified. Reviewed By: vedanuj Differential Revision: D22633312 Pulled By: apsdehal fbshipit-source-id: d1319906b28f32bd900088a62f21fa1e6bb5bc13 | 06 August 2020, 20:54:16 UTC |
7d89369 | Sasha Sheng | 06 August 2020, 20:09:58 UTC | [fix] raise value error rather than exception; fixed formating with black (#458) Summary: [fix] raise value error rather than exception; fixed formating with black - I apologize with the premature land, I kicked off transfering code before I saw the comments. - Test with ``pytest tests`` - This passes tests Question, would you be open to using git precommit hooks to run isort, black automatically? Pull Request resolved: https://github.com/facebookresearch/mmf/pull/458 Reviewed By: apsdehal Differential Revision: D22982028 Pulled By: ytsheng fbshipit-source-id: 9c07b0dec7ff3b4ac241a1796415b186bbec3a59 | 06 August 2020, 20:11:18 UTC |
faf8edd | Suzanna Sia | 06 August 2020, 04:51:32 UTC | [fix, refactor] Tuple output bug in visual_bert model (#457) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/457 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/126 [fix]: The first object returned should always be return `encoded_layers[0]` or `sequence_output`, or there will be a `tuple` error. [refactor]; Remove duplicate lines by better placing of the conditional statement `if self.output_attentions` . No actual logic changed. Reviewed By: vedanuj, apsdehal Differential Revision: D22970391 fbshipit-source-id: 9617c9b6b9ab7defe59dcad127f19a6f045b94be | 06 August 2020, 04:52:34 UTC |
6329d7c | Sasha Sheng | 06 August 2020, 04:38:58 UTC | [feat] Support epoch based training (#449) Summary: Hi y'all, this is my first PR, so let me know how you feel about my approach. :) Some logging was only added for this test plan and therefore does not exist in this PR. Let me know if you find those logging helpful. I can always add them back. A couple contributions: * This adds support for max_epochs. Using num_updates to control training loop; * If you have both num_epochs and num_updates specified, we warn you, and favor num_epochs. * This supports fractional epoch. * I think I found an issue where we are training max_updates + 1 iterations, this PR also fixes it. When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=None\ training.max_epochs=None ``` ![Screen Shot 2020-08-04 at 11 53 39 AM](https://user-images.githubusercontent.com/1545487/89333073-4feb5180-d649-11ea-9d38-8f480bec0bb2.png) When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=10\ training.max_epochs=1 ``` we favor max_epochs over max_updates, added a warning message. ![Screen Shot 2020-08-04 at 11 58 25 AM](https://user-images.githubusercontent.com/1545487/89333543-f3d4fd00-d649-11ea-8711-59e8d678cef1.png) When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=None\ training.max_epochs=1 ``` ![Screen Shot 2020-08-04 at 1 52 54 AM](https://user-images.githubusercontent.com/1545487/89274076-4637fe80-d5f5-11ea-89cc-131f167beb38.png) When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml \ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=10\ training.max_epochs=None ``` ![Screen Shot 2020-08-04 at 1 13 11 AM](https://user-images.githubusercontent.com/1545487/89270083-b04da500-d5ef-11ea-9c0e-26cbba18f3ca.png) To test fractional epochs: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=40\ training.max_epochs=0.04 ``` <img width="1073" alt="Screen Shot 2020-08-04 at 9 59 22 AM" src="https://user-images.githubusercontent.com/1545487/89323554-bb79f280-d63a-11ea-96d4-1cdafc47b7e2.png"> To test: pytest tests/trainers/test_training_loop.py ![Screen Shot 2020-08-04 at 4 58 05 PM](https://user-images.githubusercontent.com/1545487/89356885-c8ff9e80-d673-11ea-9f5c-43e9a520e508.png) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/449 Reviewed By: apsdehal Differential Revision: D22963208 Pulled By: ytsheng fbshipit-source-id: cfe9786314287c3f7f31ce2b46d602568b5ffd4c | 06 August 2020, 04:40:27 UTC |
2dc099a | Sasha Sheng | 06 August 2020, 00:17:01 UTC | [fix] run isort (#452) Summary: 76 passed, 1 skipped, 31 warnings in 91.18s Tests: `pytest tests` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/452 Reviewed By: apsdehal Differential Revision: D22959790 Pulled By: ytsheng fbshipit-source-id: 9e0480c147ccf6caff82c41ab6c41bbe9bba18e2 | 06 August 2020, 00:18:47 UTC |
5e41e02 | Sasha Sheng | 04 August 2020, 19:59:45 UTC | Fix black reformatting errors (#451) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/451 Reviewed By: vedanuj Differential Revision: D22927669 Pulled By: ytsheng fbshipit-source-id: c1fd5f5d45133004a1c2899d019daadb4d1fb3c8 | 04 August 2020, 20:01:52 UTC |
6fd7853 | Amanpreet Singh | 03 August 2020, 18:38:28 UTC | [fix] Captioning predictions for bsz 1, remove opinions in testreporter (#437) Summary: - Use object_to_byte and reverse for saving image ids - Remove opinions about shapes and gathering in the test reporter based on keys. Generalized logic now. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/437 Reviewed By: vedanuj Differential Revision: D22836831 Pulled By: apsdehal fbshipit-source-id: 4ff6b977c41b017e15b66b499fabf7883754c1ef | 03 August 2020, 18:39:56 UTC |
efa331e | Charles Bai | 03 August 2020, 18:17:30 UTC | [fix] Override of final array elements through dot element (#438) Summary: Should check if we reach the end of the dot_list when overriding config with args. This should fix issue https://github.com/facebookresearch/mmf/issues/431 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/438 Test Plan: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml \ datasets=textvqa \ model=m4c \ run_type=train_val \ training.lr_steps[1]=100 \ dataset_config.textvqa.zoo_requirements[0]="test" \ ``` Output: ``` Overriding option config to projects/m4c/configs/textvqa/defaults.yaml Overriding option datasets to textvqa Overriding option model to m4c Overriding option run_type to train_val Overriding option training.lr_steps[1] to 100 Overriding option dataset_config.textvqa.zoo_requirements[0] to test ``` Also: `pytest ./tests/configs/test_config_overrides.py` Reviewed By: apsdehal Differential Revision: D22894342 Pulled By: charlesxbai fbshipit-source-id: bbd67b0c223c571bd2f705299d4a48cf48a49655 | 03 August 2020, 18:18:58 UTC |
9ba7af6 | Hu Xu | 01 August 2020, 05:27:58 UTC | [mmf][fix]Assertion message to match if condition Summary: Correction of assertion message on the first dimension . Reviewed By: apsdehal Differential Revision: D22860233 fbshipit-source-id: d8280d81a84ee23b51512f987302b4397318b0a5 | 01 August 2020, 05:29:00 UTC |
7cdbf8c | dependabot[bot] | 31 July 2020, 22:25:45 UTC | Bump elliptic from 6.5.2 to 6.5.3 in /website (#439) Summary: Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.2 to 6.5.3. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/indutny/elliptic/commit/8647803dc3d90506aa03021737f7b061ba959ae1"><code>8647803</code></a> 6.5.3</li> <li><a href="https://github.com/indutny/elliptic/commit/856fe4d99fe7b6200556e6400b3bf585b1721bec"><code>856fe4d</code></a> signature: prevent malleability and overflows</li> <li>See full diff in <a href="https://github.com/indutny/elliptic/compare/v6.5.2...v6.5.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=elliptic&package-manager=npm_and_yarn&previous-version=6.5.2&new-version=6.5.3)](https://help.github.com/articles/configuring-automated-security-fixes) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/439 Reviewed By: apsdehal Differential Revision: D22858238 Pulled By: vedanuj fbshipit-source-id: 040488c2037bc4235635ffffc102e341da7140db | 31 July 2020, 22:27:24 UTC |
cb2adc6 | Amanpreet Singh | 30 July 2020, 20:25:19 UTC | [feat,fix] Logger overhaul, shift to Python logger (#436) Summary: - All logging uses python based logging after this - Colored logging support inspired from d2 - Wherever touched moved to format strings - Adds options for colored logging - Moves torch version and cuda devices to top - Moves all print and writer statements to logger - Also, fixes the long failing windows tests for logger What are the benefits of doing this? - All libraries which use python logging will receives our logs now and we receive theirs - Proper division between stderr and stdout Pull Request resolved: https://github.com/facebookresearch/mmf/pull/436 Test Plan: Tested locally Reviewer suggestions: This PR needs to be reviewed somewhat carefully to make sure I don't miss anything and as it touches all of the MMF. Reviewed By: vedanuj Differential Revision: D22819619 Pulled By: apsdehal fbshipit-source-id: 6675ab90567c0d192c3023ede1cdb183dd103ea6 | 30 July 2020, 20:26:48 UTC |
3fc0cdf | Vedanuj Goswami | 28 July 2020, 01:45:11 UTC | [feat] Add Movie+Mcan grid features to zoo (#428) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/428 Added the grid features used for movie+mcan model to dataset zoo for automated downloads. Reviewed By: apsdehal Differential Revision: D22721729 fbshipit-source-id: 51757819404510a48b984f5437ac1fc023c59e65 | 28 July 2020, 01:46:12 UTC |
9f28973 | Amanpreet Singh | 27 July 2020, 20:54:07 UTC | [fix] Move anomaly detection to an option (#434) Summary: - As per PyTorch docs anomaly detection should only be turned on for debugging purposes and can slow down things - This warning wasn't there when the code was initially added so we missed this. Setting this to false will drastically improve performance of all our models. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/434 Test Plan: Tested locally on VisualBERT with the setting off Reviewed By: vedanuj Differential Revision: D22768897 Pulled By: apsdehal fbshipit-source-id: f990406fc1b8e294aeb726013eeaef44c453b4c4 | 27 July 2020, 20:55:28 UTC |
c75f245 | Vedanuj Goswami | 22 July 2020, 04:57:47 UTC | [feat] Add Base transformer class for modular multimodal transformers (#398) Summary: - Adds a base transformer model class with modular components for constructing a transformer based multi modal model. The modularity is divided as : ``` self.build_encoders() self.build_embeddings() self.build_transformer() self.build_heads() self.build_losses() ``` - Base model automatically loads an encoder specified by `transformer_base` in config. For example it can be `bert-base-uncased`, `roberta-base` etc. - Ability to preprocess the input sample_list in `preprocess_sample ` according to model's requirement and also postprocess the output from the transformer according to requirement. For pretraining, we can add cls head and calculate loss on the output in the `postprocess_output ` method. For classification we can calculate scores etc. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/398 Test Plan: - Tested with creating a new model using the base class and running on Hateful Memes, VQA2 Reviewed By: apsdehal Differential Revision: D22480254 Pulled By: vedanuj fbshipit-source-id: 7eec160941f2146c8822d7590d50acb7c74b6904 | 22 July 2020, 04:58:55 UTC |
18bcbd8 | dependabot[bot] | 22 July 2020, 04:20:32 UTC | [chores] Bump lodash from 4.17.15 to 4.17.19 in /website (#423) Summary: Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/lodash/lodash/releases">lodash's releases</a>.</em></p> <blockquote> <h2>4.17.16</h2> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/lodash/lodash/commit/d7fbc52ee0466a6d248f047b5d5c3e6d1e099056"><code>d7fbc52</code></a> Bump to v4.17.19</li> <li><a href="https://github.com/lodash/lodash/commit/2e1c0f22f425e9c013815b2cd7c2ebd51f49a8d6"><code>2e1c0f2</code></a> Add npm-package</li> <li><a href="https://github.com/lodash/lodash/commit/1b6c282299f4e0271f932b466c67f0f822aa308e"><code>1b6c282</code></a> Bump to v4.17.18</li> <li><a href="https://github.com/lodash/lodash/commit/a370ac81408de2da77a82b3c4b61a01a3b9c2fac"><code>a370ac8</code></a> Bump to v4.17.17</li> <li><a href="https://github.com/lodash/lodash/commit/1144918f3578a84fcc4986da9b806e63a6175cbb"><code>1144918</code></a> Rebuild lodash and docs</li> <li><a href="https://github.com/lodash/lodash/commit/3a3b0fd339c2109563f7e8167dc95265ed82ef3e"><code>3a3b0fd</code></a> Bump to v4.17.16</li> <li><a href="https://github.com/lodash/lodash/commit/c84fe82760fb2d3e03a63379b297a1cc1a2fce12"><code>c84fe82</code></a> fix(zipObjectDeep): prototype pollution (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4759">#4759</a>)</li> <li><a href="https://github.com/lodash/lodash/commit/e7b28ea6cb17b4ca021e7c9d66218c8c89782f32"><code>e7b28ea</code></a> Sanitize sourceURL so it cannot affect evaled code (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4518">#4518</a>)</li> <li><a href="https://github.com/lodash/lodash/commit/0cec225778d4ac26c2bac95031ecc92a94f08bbb"><code>0cec225</code></a> Fix lodash.isEqual for circular references (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4320">#4320</a>) (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4515">#4515</a>)</li> <li><a href="https://github.com/lodash/lodash/commit/94c3a8133cb4fcdb50db72b4fd14dd884b195cd5"><code>94c3a81</code></a> Document matches* shorthands for over* methods (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4510">#4510</a>) (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4514">#4514</a>)</li> <li>Additional commits viewable in <a href="https://github.com/lodash/lodash/compare/4.17.15...4.17.19">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~mathias">mathias</a>, a new releaser for lodash since your current version.</p> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=lodash&package-manager=npm_and_yarn&previous-version=4.17.15&new-version=4.17.19)](https://help.github.com/articles/configuring-automated-security-fixes) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/423 Reviewed By: vedanuj Differential Revision: D22657049 Pulled By: apsdehal fbshipit-source-id: 924e15b18c730b2ddbd3fb12b431536463d8bff4 | 22 July 2020, 04:21:30 UTC |
9c2662f | Vedanuj Goswami | 22 July 2020, 03:29:19 UTC | [fix] Rename transformer_bbox_processor as bbox_processor already exists (#424) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/424 Fix `bbox_processor` bug due to duplicate name of processors. Rename to `transformer_bbox_processor` for transformer specific bbox processor Reviewed By: apsdehal Differential Revision: D22655123 fbshipit-source-id: d0c8e070afaf726086c1d34fd4040b0eacaf8615 | 22 July 2020, 03:30:37 UTC |
3d0c2ae | Amanpreet Singh | 22 July 2020, 02:37:43 UTC | [feat] OKVQA dataset implementation (#420) Summary: Adds support for OKVQA dataset in MMF. Automatic downloads are only supported at image level, feature support maybe added later. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/420 Test Plan: A config has been added to test on MMBT with Images. ``` mmf_run config=projects/mmbt/configs/okvqa/with_images.yaml run_type=train_val dataset=okvqa model=mmbt training.batch_size=64 training.stdout_capture=False training.seed=1 ``` Reviewed By: vedanuj Differential Revision: D22608435 Pulled By: apsdehal fbshipit-source-id: 775d19d93063ce3b8cbcb875e17ce4ef2dc2f2f7 | 22 July 2020, 02:38:52 UTC |
4398068 | Stanislau Hlebik | 18 July 2020, 00:07:23 UTC | remediation of S205607 fbshipit-source-id: 798decc90db4f13770e97cdce3c0df7d5421b2a3 | 18 July 2020, 00:18:47 UTC |
a7ba97f | Stanislau Hlebik | 18 July 2020, 00:07:23 UTC | remediation of S205607 fbshipit-source-id: 5113fe0c527595e4227ff827253b7414abbdf7ac | 18 July 2020, 00:18:47 UTC |
c4d9f0b | Vedanuj Goswami | 17 July 2020, 22:17:44 UTC | [WIP][fix] Run actions on PRs as well (#419) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/419 Reviewed By: apsdehal Differential Revision: D22603770 Pulled By: vedanuj fbshipit-source-id: 774b42a1dfe6209844d92ecc6ab621c4463b9e06 | 17 July 2020, 22:18:50 UTC |
e11e2b5 | Vedanuj Goswami | 17 July 2020, 05:53:48 UTC | [fix] Fix windows logger tests, HM Readme (#418) Summary: Fixes https://github.com/facebookresearch/mmf/issues/417 , Fixes windows test error due to permissions removing temp dir. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/418 Reviewed By: apsdehal Differential Revision: D22592532 Pulled By: vedanuj fbshipit-source-id: 28e0e80ad52556716279c5334acdffddf0340cc8 | 17 July 2020, 05:54:55 UTC |
466155e | Amanpreet Singh | 17 July 2020, 01:58:44 UTC | [fix] Checkpoint resume dir and value in sweeps scripts (#412) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/412 - Checkpoint resume wasn't enabled in fblearner script. - Checkpoint.resume should directly checked truthy. This will allow users to pass `checkpoint.resume=1` and avoid confusion. Reviewed By: vedanuj Differential Revision: D22566067 fbshipit-source-id: 18975236769c7cb554b7f2e28c7cc8d6a8b82ceb | 17 July 2020, 01:59:56 UTC |
1e1edce | Amanpreet Singh | 17 July 2020, 01:40:39 UTC | [feat] FB Defaults (#413) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/413 Separate out fb defaults into a separate config for future use. Reviewed By: vedanuj Differential Revision: D22566143 fbshipit-source-id: b21b697f1ef9686127200a94f96d6db7b464869e | 17 July 2020, 01:41:47 UTC |
93c0300 | Vedanuj Goswami | 17 July 2020, 00:16:54 UTC | [refactor] Add transformer bbox processor (#397) Summary: - Refactor `bbox` position processing for transformer based models - Fix feature path in masked coco dataset Pull Request resolved: https://github.com/facebookresearch/mmf/pull/397 Test Plan: - Test with VilBERT model on VQA2, Masked COCO Reviewed By: apsdehal Differential Revision: D22479935 Pulled By: vedanuj fbshipit-source-id: aa310d1d963facd058ba6a5322031514b15cb01c | 17 July 2020, 00:18:26 UTC |
d028012 | Amanpreet Singh | 16 July 2020, 22:13:15 UTC | [feat] Add support for BatchProcessors and identity feat encoder (#411) Summary: - BatchProcessor type to initialize batche based processors - Also add support for identity feature encoder Pull Request resolved: https://github.com/facebookresearch/mmf/pull/411 Reviewed By: vedanuj Differential Revision: D22565866 Pulled By: apsdehal fbshipit-source-id: 39fb8af1bf118d0c92eb2e7c7609140def0e593d | 16 July 2020, 22:14:23 UTC |
f2ee179 | Amanpreet Singh | 16 July 2020, 20:05:54 UTC | [refactor] Release some restrictions on supported dataset types (#410) Summary: - Allows datasets which are not inherited from BaseDataset to be returned from the DatasetBuilder - This will allow us to use IterableDataset which is required for some internal dataset - Batches can be directly returned now - Sampler doesn't need to have set_epoch method - Move fast read to VQABuilder as that's the only builder using it currently. - If prepare_batch is not implemented, throw a warning instead of error. User needs to implement it themselves now - `update_registry_for_model` is optional as well. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/410 Reviewed By: vedanuj Differential Revision: D22563932 Pulled By: apsdehal fbshipit-source-id: df68a8b330fcfd598d0428d848e90084f67965bb | 16 July 2020, 20:07:42 UTC |
4ba79d5 | Amanpreet Singh | 16 July 2020, 17:34:57 UTC | [fix] COCO dataset, checksum and missing annotation dbs (#416) Summary: - Also, fixes https://github.com/facebookresearch/mmf/issues/415 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/416 Test Plan: - Tested BUTD on COCO Reviewed By: vedanuj Differential Revision: D22573643 Pulled By: apsdehal fbshipit-source-id: 1758e60ab4263e88438c748b7275685b98e3100e | 16 July 2020, 17:36:33 UTC |
e1adfc4 | Shir Gur | 16 July 2020, 16:51:36 UTC | [fix] Fix image_database (#403) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/403 - Removes adding ".XXX" extensions in ```_get_attrs``` - Change ```from_path``` to use full image path Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/123 Reviewed By: apsdehal Differential Revision: D22473774 Pulled By: shirgur fbshipit-source-id: 876ea92d72527f89eabdc4f925cb9408216bef36 | 16 July 2020, 16:53:02 UTC |
a79bb99 | Vedanuj Goswami | 16 July 2020, 02:06:41 UTC | [feat] Add support for manifold image databases and exp save directory (#407) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/407 - Adds support for manifold image database - Adds manifold handler - Fix checkpoint and logger to support creating save_dir in manifold - Created new bucket for FAIR MMF : `manifold://fair_mmf/tree/data` Reviewed By: apsdehal Differential Revision: D22544389 fbshipit-source-id: e5cfc660b3e815d1495f1968dfb8695d2b3c58ea | 16 July 2020, 02:08:19 UTC |
42019cc | Amanpreet Singh | 15 July 2020, 19:46:48 UTC | [fix] Isort version in linux as well (#408) Summary: - Also, prints the version of the linters used now Pull Request resolved: https://github.com/facebookresearch/mmf/pull/408 Test Plan: To be tested on CI Reviewed By: vedanuj Differential Revision: D22551651 Pulled By: apsdehal fbshipit-source-id: 4d5d5cb44c9720de60c2d395c102bc55346b2574 | 15 July 2020, 19:48:16 UTC |
11174ad | Amanpreet Singh | 15 July 2020, 05:53:18 UTC | [fix] Isort version same as pre-commit to avoid CI failures (#406) Summary: - With recent update of isort, it is changing alias imports to custom format, which is breaking our CI builds - Later, we should look into how we can move to the latest version of isort Pull Request resolved: https://github.com/facebookresearch/mmf/pull/406 Test Plan: To be tested on CI Reviewed By: vedanuj Differential Revision: D22542173 Pulled By: apsdehal fbshipit-source-id: 35e5cea04e8f6d61ac37f6c1482699dda9205c12 | 15 July 2020, 05:54:30 UTC |
4a9862b | Ivan Petrov Evtimov | 14 July 2020, 02:29:08 UTC | Checkpoint mmf training only last n intervals (#401) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/401 Per task 67509713, modify mmf model training to checkpoint only last n checkpoints Reviewed By: apsdehal Differential Revision: D22406213 fbshipit-source-id: 130fb5af458fce44dada8b28846665b039085e00 | 14 July 2020, 02:30:21 UTC |
6fd6cb9 | Amanpreet Singh | 13 July 2020, 23:13:05 UTC | [feat] Support internal website for MMF (#400) Summary: - Adds package dependency on MMF internal website docusaurus package Pull Request resolved: https://github.com/facebookresearch/mmf/pull/400 Test Plan: Tested locally See relevant diff: D22510353 Reviewed By: vedanuj Differential Revision: D22510683 Pulled By: apsdehal fbshipit-source-id: f7634b7579124610702647ea68b89d6ac8b38f13 | 13 July 2020, 23:14:10 UTC |
4ef79b0 | Amanpreet Singh | 09 July 2020, 21:19:45 UTC | [fix] Distributed seed should be set different for each process (#394) Summary: - This causes issues for things which should be random in different processes instead of being same like RandomSampler Pull Request resolved: https://github.com/facebookresearch/mmf/pull/394 Test Plan: - Tested on VisualBERT with VQA2 cc ronghanghu Reviewed By: vedanuj Differential Revision: D22450741 Pulled By: apsdehal fbshipit-source-id: b8d86e98840be7e2e7d7893930f1e6fb3406c3a7 | 09 July 2020, 21:20:53 UTC |
f5fc894 | Amanpreet Singh | 08 July 2020, 22:22:40 UTC | [refactor] Make build method public in annotation db and mmf dataset (#393) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/393 Test Plan: Tested on m4c Reviewed By: vedanuj Differential Revision: D22438277 Pulled By: apsdehal fbshipit-source-id: 1ce2af376f62d6886346e1ed05ef0dac145bb83b | 08 July 2020, 22:25:14 UTC |
948f6fe | Shir Gur | 07 July 2020, 07:11:41 UTC | [feat] Add pre/post processing for predictions (#122) Summary: Adding ```on_prediction_start```and ```on_prediction_end``` for pre/post processing of the predictions Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/122 Reviewed By: vedanuj, apsdehal Differential Revision: D22397973 Pulled By: shirgur fbshipit-source-id: 957b364b393c92ec90f1b50c12a827bbfd3c7bc1 | 07 July 2020, 07:13:03 UTC |
b77e8b2 | Amanpreet Singh | 07 July 2020, 05:28:24 UTC | [tests] Add test for checking if shasum is valid for zoo (#380) Summary: - Uses a regex to make sure all of the checksums are valid sha256sums in the zoo configs - Also refactor test for zoos to be reusable Pull Request resolved: https://github.com/facebookresearch/mmf/pull/380 Reviewed By: vedanuj Differential Revision: D22406875 Pulled By: apsdehal fbshipit-source-id: 063e222372fcf03d6d67a2273f6439c1a5db7fd5 | 07 July 2020, 05:29:42 UTC |
08001d3 | Amanpreet Singh | 07 July 2020, 04:21:32 UTC | [fix] Processor id in sidebar and deploy (#391) Summary: - Deploy is broken due to wrong id in the processor docs Pull Request resolved: https://github.com/facebookresearch/mmf/pull/391 Reviewed By: vedanuj Differential Revision: D22406872 Pulled By: apsdehal fbshipit-source-id: 1006495e90d1481cba096aed0b0b88168b0e0367 | 07 July 2020, 04:23:53 UTC |
31e6d7b | Suzanna Sia | 07 July 2020, 02:02:42 UTC | [fix, refactor] Truncate bug and readable convert to index (#383) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/383 Fix a bug in `truncate_seq_pair` where if `tokens_b` is not `None`, the `max_length` should be `max_length-3` because there are 3 special tokens, [CLS] tokens_a [SEP] tokens_b [SEP]. Refactor `tokens` and `segment_ids` in the same sequential form as `lm_label_ids` for readability, does not change the code logic. Reviewed By: apsdehal Differential Revision: D22344634 fbshipit-source-id: 7aec985843e225587df74c3b4abe587abeb7f394 | 07 July 2020, 02:04:15 UTC |
3d78fb7 | Suzanna Sia | 07 July 2020, 01:47:05 UTC | [refactor] Gathering of distributed tensors based on number of dimensions (#384) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/384 Refactor to automatically check the dimensions and apply the appropriate reshaping before gathering distributed tensors. To make writing custom reports more convenient when new keys are added. � Did not change any of the actual logic. Makes the reshaping explicit depending on the number of the dimensions. Gathering distributed tensors increases the number of dimensions by 1 and we reshape accordingly. Will fail more gracefully if the tensor shape before `gather_tensor` is 0 or greater than 2. Reviewed By: vedanuj, apsdehal Differential Revision: D22299088 fbshipit-source-id: dbe71227a147bacf426fe3b6cc5c7eea463f8c7e | 07 July 2020, 01:48:01 UTC |
4a5c707 | Amanpreet Singh | 06 July 2020, 23:15:42 UTC | [fix] Epoch counting in trainer (#389) Summary: - Current epoch counting has been broken and overestimates the count Pull Request resolved: https://github.com/facebookresearch/mmf/pull/389 Test Plan: Test on m4c/textvqa in distributed settings as well Reviewed By: vedanuj Differential Revision: D22399463 Pulled By: apsdehal fbshipit-source-id: 7714bb91cce5581fffcfadbc464325d9a4856517 | 06 July 2020, 23:16:55 UTC |
2dc3190 | Vedanuj Goswami | 06 July 2020, 19:40:37 UTC | [docs] Fix MoViE+MCAN docs (#388) Summary: Some doc fixes Pull Request resolved: https://github.com/facebookresearch/mmf/pull/388 Reviewed By: apsdehal Differential Revision: D22393314 Pulled By: vedanuj fbshipit-source-id: 5f91044a3093177074943a92f85b8466857205a4 | 06 July 2020, 19:42:16 UTC |
c91d660 | Amanpreet Singh | 06 July 2020, 17:04:03 UTC | [fix] Call load_state_dict in on_init_start and call it in trainer.load (#385) Summary: - This fixes the issue with loading distributed or data parallel models using checkpoint as model would parallelize but still won't have checkpoint loaded - A missing call to on_init_start has also been added Pull Request resolved: https://github.com/facebookresearch/mmf/pull/385 Test Plan: Tested by Omkar Reviewed By: vedanuj Differential Revision: D22370051 Pulled By: apsdehal fbshipit-source-id: 2176d4f410f9313a9bcf18ac5a0eec9338df2fc2 | 06 July 2020, 17:05:16 UTC |
47da75a | Amanpreet Singh | 03 July 2020, 03:18:20 UTC | [feat,fix] New textvqa features, retrained model, some fixes (#375) Summary: - Adds new features for TextVQA based on maskrcnn benchmark script which now also contains extra info as required to fix https://github.com/facebookresearch/mmf/issues/328 - Adds retrained model using these new features on VMB, on test-std this model gets 41.16% higher than the previous model - Fixes the issue with the answer processor and in general predictions on TextVQA for multiple models (LoRRA and M4C) - Fixes joint_stvqa script to properly load features for STVQA. The fix is hacky as of now and requires some refactoring in datasets to properly support dataset class attributes - Fixes the broken extract features scripts to use latest code and functions as well downlaod functionalities Pull Request resolved: https://github.com/facebookresearch/mmf/pull/375 Test Plan: Tested extensively on TextVQA with LoRRA as well as M4C Reviewed By: ronghanghu Differential Revision: D22321533 Pulled By: apsdehal fbshipit-source-id: fa97a7d56a333c6e1b6ffb1103e69cbaf7b1d783 | 03 July 2020, 03:19:36 UTC |
b7287eb | Amanpreet Singh | 02 July 2020, 19:05:46 UTC | [fix] Correct reference of tensorboard argument to clip gradient (#382) Summary: - This was causing the training to fail after recent trainer refactor in https://github.com/facebookresearch/mmf/issues/332 - Will properly fix https://github.com/facebookresearch/mmf/issues/378 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/382 Reviewed By: vedanuj, ronghanghu Differential Revision: D22353205 Pulled By: apsdehal fbshipit-source-id: 8f719542598b091b346496f702ca1d898ecf7d02 | 02 July 2020, 19:07:17 UTC |
95c6287 | Vedanuj Goswami | 01 July 2020, 18:14:51 UTC | [fix] Fix should_log in training_loop (#379) Summary: Fixes https://github.com/facebookresearch/mmf/issues/378 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/379 Reviewed By: apsdehal Differential Revision: D22334049 Pulled By: vedanuj fbshipit-source-id: faa51b4c8cf865850851df334b9971de92b641d1 | 01 July 2020, 18:16:08 UTC |
b823f25 | Amanpreet Singh | 01 July 2020, 17:59:27 UTC | [fix] VisualBERT regression of pooler; provide pooler strategy option (#377) Summary: - In VQA2 and general question answering, original codebase has a different pooler strategy of taking hidden state from second last token - The regression was causing issue with mismatch of accuracies on pretraining it right paper's checkpoint Pull Request resolved: https://github.com/facebookresearch/mmf/pull/377 Test Plan: Testing on the best checkpoint and estimated the accuracy to be same. Reviewed By: vedanuj Differential Revision: D22328981 Pulled By: apsdehal fbshipit-source-id: bcd289181289e500c29d3fd73e1a8600f0ce5b3d | 01 July 2020, 18:00:37 UTC |
d1d10cb | Vedanuj Goswami | 01 July 2020, 07:07:15 UTC | [feat] Add MoViE+MCAN model (#345) Summary: - Adding the MoVie+MCAN model (VQA 2020 Challenge Winner) TODOs: - [x] Add typings to movie_mcan.py - [x] Upload the annotations to zoo - [x] Upload best model to zoo - [x] Cleanup feature_readers.py - [x] Update Readme Pull Request resolved: https://github.com/facebookresearch/mmf/pull/345 Reviewed By: apsdehal Differential Revision: D22214947 Pulled By: vedanuj fbshipit-source-id: 99da94efdbf34b74338b62299570a299860cb6d1 | 01 July 2020, 07:08:33 UTC |
a1a1a55 | Amanpreet Singh | 01 July 2020, 06:41:20 UTC | [fix] Reloads in spawned processes shouldn't setup imports again (#374) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/355 partially - In general, only setup_imports in the main processor - This means that the child processes are using only shallow registry which is fine as setup imports can cause issues in file not related with syntax error - Overall, this can't be fixed completely as by the nature of the spawn, the child will still reload all of the code and if a syntax error is there it will still break the worker. General suggestion is to keep two copies of the code, one for development and one for running and keep them in sync via GitHub Pull Request resolved: https://github.com/facebookresearch/mmf/pull/374 Test Plan: Tested with command and reproduction steps provided by Ronghang Reviewed By: vedanuj Differential Revision: D22322313 Pulled By: apsdehal fbshipit-source-id: 6a8e388df1244ff450f726bc66535ff53900da14 | 01 July 2020, 06:42:39 UTC |
208579e | Amanpreet Singh | 30 June 2020, 22:17:38 UTC | [docs] Tutorial on SLURM sweeps (#359) Summary: - Runs prettier on rest of docs as well - Point original readme to the new doc Pull Request resolved: https://github.com/facebookresearch/mmf/pull/359 Reviewed By: vedanuj Differential Revision: D22310372 Pulled By: apsdehal fbshipit-source-id: 4b3e61097fba319d8ea05c7e7d2cfda3d7074593 | 30 June 2020, 22:18:54 UTC |
f021a7a | Amanpreet Singh | 30 June 2020, 21:53:03 UTC | [fix] Resolve config when it is created (#371) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/370. See it for details Pull Request resolved: https://github.com/facebookresearch/mmf/pull/371 Test Plan: Tested with visual bert run Reviewed By: vedanuj Differential Revision: D22297254 Pulled By: apsdehal fbshipit-source-id: 1a6549774dacf2f8cda492c9d2092b2ce42ccba7 | 30 June 2020, 21:54:11 UTC |
025f05d | Durgesh | 30 June 2020, 16:28:20 UTC | [fix] Installation Link in hateful_memes/README.md (#368) Summary: Broken Link in hateful_memes/README.md for mmf installation Fix Pull Request resolved: https://github.com/facebookresearch/mmf/pull/368 Reviewed By: apsdehal Differential Revision: D22291475 Pulled By: vedanuj fbshipit-source-id: 239ab1891576d234c8a146826e3999298474cc7a | 30 June 2020, 16:29:19 UTC |
b620c01 | Vedanuj Goswami | 30 June 2020, 16:25:26 UTC | [fix] Rename imdb to annotation_db (#372) Summary: Renames remaining `imdb` to `annotation_db` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/372 Reviewed By: apsdehal Differential Revision: D22304951 Pulled By: vedanuj fbshipit-source-id: 47ead15b4741d1c0628782951cc3232992d444c7 | 30 June 2020, 16:26:28 UTC |
48631d8 | Vedanuj Goswami | 30 June 2020, 03:57:09 UTC | [feat, refactor] Trainer class refactor (#332) Summary: PR refactors the trainer class providing flexibility to extend from the BaseTrainer class and customize it. We also introduce a callback mechanism that can be executed at different events during training, validation or inference. Hence separates most of the logic of core training/optimization from other logistics. - Create an abstract base trainer class, and core functionalities like loading dataset, model, optimizer, metrics are abstract methods that has to be implemented by derived classes. - Create a new `MMFTrainer` class which keeps the same behavior as the current trainer but derived from the `BaseTrainer`. - Move core logic of training loop, evaluation loop and prediction loop to Mixin classes and can be easily used in trainer classes. - Create a callback mechanism similar to `PytorchLightning` with different types of events when callbacks can happen. With this it will be easier to have `PytorchLightning` supported trainer classes for `mmf` - Callbacks are executed before or after events like training, validation, test, batch, epoch etc. - Typing introduced in the trainer classes - There can be some additional refactoring around how to include some additional callbacks like Early Stopping in the hooks, but that can be done in future diffs. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/332 Test Plan: - Tested training, inference, validation runs, checked if events are happening in correct order with the new callback mechanism. Reviewed By: apsdehal Differential Revision: D22112333 Pulled By: vedanuj fbshipit-source-id: 07dbd42962691cf6b64bca2d53dc98f17150c853 | 30 June 2020, 03:58:20 UTC |