Skip to main content

Browse the archive

https://github.com/facebookresearch/pythia
31 March 2023, 09:22:40 UTC
sort by:
Revision Author Date Message Commit Date
584e26d Vedanuj Goswami 08 September 2020, 22:59:00 UTC [chores] Upgrade to pytorch 1.6 and torchvision 0.7 (#546) Summary: - Upgrading to pytorch 1.6, which comes with new features required for upcoming torchscript changes - Optimizer keys are no longer non-deterministic in PT1.6, check pytorch [PR](https://github.com/pytorch/pytorch/pull/37347 ). Remove `skip_keys` check from checkpoint optimizer tests. - Fixed nucleus sampling tests - Update docs to reflect support for pytorch 1.6 - Update CI tests, CircleCI and Actions to use pytorch 1.6, torchvision 0.7 Test Plan : - Tested with visual bert and mmbt models on Hateful Memes - Running internally at FB (runs on pytorch master) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/546 Reviewed By: apsdehal Differential Revision: D23558059 Pulled By: vedanuj fbshipit-source-id: 3d82978276462a3f3dea564ac33581240e284d03 08 September 2020, 23:00:27 UTC
9e83320 steph-en-m 08 September 2020, 18:57:05 UTC [feat] Add descriptive error message for missing processor in registry (#548) Summary: This PR resolves https://github.com/facebookresearch/mmf/issues/455 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/548 Reviewed By: vedanuj Differential Revision: D23559767 Pulled By: apsdehal fbshipit-source-id: 59e4e4e5d7dd9eaf88edf81b6906d5d8768f83a2 08 September 2020, 18:58:48 UTC
3b8e26b Amanpreet Singh 06 September 2020, 01:59:53 UTC [fix] Update HM docs for resume_pretrained=False during evaluation (#545) Summary: - Update README and docs Pull Request resolved: https://github.com/facebookresearch/mmf/pull/545 Reviewed By: vedanuj Differential Revision: D23545636 Pulled By: apsdehal fbshipit-source-id: bd23195e39584725b447d7be8ccedd0281a41d76 06 September 2020, 02:00:48 UTC
efe4023 steph-en-m 04 September 2020, 19:56:04 UTC [fix] modify mmf_convert_hm to create a copy of a file (#532) Summary: This PR resolves https://github.com/facebookresearch/mmf/issues/528 to ensure `mmf_convert_hm` preserves the original file Pull Request resolved: https://github.com/facebookresearch/mmf/pull/532 Test Plan: - tested it on the hateful memes dataset on colab and the original file is preserved. Reviewed By: apsdehal Differential Revision: D23545590 Pulled By: vedanuj fbshipit-source-id: 136f6bd9a526d4ed91fc27b4420160fbe9ff02e0 04 September 2020, 19:57:07 UTC
637a5bc Sasha Sheng 04 September 2020, 04:53:35 UTC [feat] added split dataset feature (#470) Summary: * allow user to specify x% of dataset to split from train, example yaml shown below: ``` textvqa: zoo_requirements: - textvqa.defaults - textvqa.ocr_en split_train: val: 0.2 test: 0.1 seed: 42 features: train: - textvqa/defaults/features/open_images/detectron.lmdb,textvqa/ocr_en/features/ocr_en_frcn_features.lmdb processors: ``` * To test: `pytests test` * See logs, 24221 is 0.7 of 34602 which is the total size of the train after calculation, etc. ![Screen Shot 2020-08-10 at 11 47 12 PM](https://user-images.githubusercontent.com/1545487/89866133-e052f600-db63-11ea-80e0-4f2c6b1a7acc.png) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/470 Reviewed By: apsdehal Differential Revision: D23098846 Pulled By: ytsheng fbshipit-source-id: 9498ae0eda8001716f8d8cf77724904a4e88a495 04 September 2020, 04:54:47 UTC
1e64628 rizavelioglu 03 September 2020, 19:43:59 UTC [fix] Typo in pretrain train+val config of visual bert (#541) Summary: fix typo in the name of the features files Pull Request resolved: https://github.com/facebookresearch/mmf/pull/541 Reviewed By: vedanuj Differential Revision: D23504310 Pulled By: apsdehal fbshipit-source-id: 9e3055a5150c98b8535edf3389804f6ecf509357 03 September 2020, 19:45:50 UTC
7a35e17 Amanpreet Singh 03 September 2020, 17:15:14 UTC [fix] Answer processor issues in MaskedVQA2Dataset (#529) Summary: - Adds add_answer option to default config - Set it to default in the dataset - Add default answer processor config Pull Request resolved: https://github.com/facebookresearch/mmf/pull/529 Test Plan: Tested on full masked_vqa2 config under VisualBERT Reviewed By: vedanuj Differential Revision: D23470296 Pulled By: apsdehal fbshipit-source-id: 8b8aacea829f2c4de392ab906de571fe8ec8d93d 03 September 2020, 17:17:03 UTC
98964eb Amanpreet Singh 03 September 2020, 16:58:55 UTC [fix] Add missing init files to new folders (#531) Summary: - Without this, these folders are not included when installed through pip Pull Request resolved: https://github.com/facebookresearch/mmf/pull/531 Test Plan: Tested in separate repo Reviewed By: vedanuj Differential Revision: D23485913 Pulled By: apsdehal fbshipit-source-id: cc49fd7763016aa58ac8aa6dc47cd2e8a40fdf39 03 September 2020, 17:00:05 UTC
24dedb1 Amanpreet Singh 01 September 2020, 23:08:45 UTC [chores] Update contributing guidelines, issue templates, installation (#524) Summary: - Contributing guidelines now mention pre commit, our style guidelines and general PR suggestions - Adds issue template for a catch all category and enforce template always - Adds Pull request template - Updates installation docs to mention conda environment, windows install, installing directly from github with pip, and contributing guidelines along with recommended, optional tags for better clarity Pull Request resolved: https://github.com/facebookresearch/mmf/pull/524 Test Plan: Tested locally by building the website Reviewed By: vedanuj Differential Revision: D23407113 Pulled By: apsdehal fbshipit-source-id: 2da0bfe91838b56c039c8a34bda94140dd5f7c62 01 September 2020, 23:10:03 UTC
d6fc967 Vedanuj Goswami 01 September 2020, 05:43:16 UTC [fix] Fix lxmert model bbox processing (#526) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/520 and use `transformer_bbox_processor` processor - `transformer_bbox_processor` will now work when bbox features already have area norm Pull Request resolved: https://github.com/facebookresearch/mmf/pull/526 Test Plan: Run lxmert model for pretraining and finetuning on vqa2 Reviewed By: apsdehal Differential Revision: D23414310 Pulled By: vedanuj fbshipit-source-id: 0fa1848448ba193e0924966ae840ea225bb718ca 01 September 2020, 05:44:20 UTC
bf148a8 Grzegorz Jacenków 29 August 2020, 17:31:04 UTC [docs] Fix tutorial for the Hateful Memes Challenge (#523) Summary: * Updates the documentation with a working code example. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/523 Reviewed By: vedanuj Differential Revision: D23407535 Pulled By: apsdehal fbshipit-source-id: e431f1a1ef6ee0105101d28701cec9bbdf9881b0 29 August 2020, 17:32:11 UTC
cb3ecd1 Amanpreet Singh 28 August 2020, 22:17:09 UTC [fix] Remove all .cuda calls from mmf so models can run on CPU (#522) Summary: - VisualBERT and VILBERT Pull Request resolved: https://github.com/facebookresearch/mmf/pull/522 Test Plan: Tested locally with and without GPU Reviewed By: vedanuj Differential Revision: D23404310 Pulled By: apsdehal fbshipit-source-id: e1e774e86d3726efd31271de1fe565b449739f2b 28 August 2020, 22:18:47 UTC
57ed846 Suzanna Sia 28 August 2020, 18:07:04 UTC [fix] PathManager in sweep script as part of Manifold Migration (#498) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/498 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/132 Adopting `PathManager` for various file io utilities as part of Manifold migration. The default checkpoint_dir is still gluster, because I'm assuming not everyone has a manifold directory. To use a manifold_dir, we would sent it as an argument `--checkpoints_dir <manifold_save_dir>` via cli. Reasons for all the if-else statements within `fblearner.py` for manifold vs gluster, * this diff actually slows down submitting jobs to flow (because of the PathManager dependency). * It also requires a custom built kernel with torch instead of a regular bento kernel to run the flow script. Hen Other changes * simplified the logic of `has_started` * better logging of run in progress and what to do * check for `config.yaml` instead of `train.log` to determine if run exists Reviewed By: apsdehal Differential Revision: D23236008 fbshipit-source-id: 0aff4684934c82e0a3cc6f55143d55e5a4b6e719 28 August 2020, 18:08:10 UTC
6d98b6c Vedanuj Goswami 28 August 2020, 15:49:43 UTC [fix] Fix grad accumulation logic for Iterable datasets (#521) Summary: PR https://github.com/facebookresearch/mmf/issues/367 breaks training with `IterableDataset` as `len(self.train_loader) == 0` since `__len__` is not implemented for every `IterableDataset`. This PR mitigates this regression and : - Sets `num_remaining_batches` to be equal to (number of updates remaining x update frequency) for `IterableDataset` case. For non `IterableDataset` the old behavior is maintained (`len(self.train_loader)`) - Modifies the logic to avoid the need for checking if iterator has finished - Fixes another instance where we used to check `len(self.train_loader)` for setting number of updates when `max_epochs` is specified. Now in case of `IterableDataset` it will fall back to using `max_updates`. Added warning. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/521 Test Plan: - Tested internally on FB datasets - Tested on Hateful Memes Reviewed By: apsdehal Differential Revision: D23395374 Pulled By: vedanuj fbshipit-source-id: 415358455d12bdcb1d7ba87dd304c203c6cd990e 28 August 2020, 15:51:08 UTC
3f313b6 Amanpreet Singh 28 August 2020, 00:27:50 UTC [feat] MultiClassFromFile Processor along with Violations (#519) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/519 - Adds violations dataset - Add processor for MultiClassFromFile - Adds batch processor for MutliClassFeaturesWithText Reviewed By: vedanuj Differential Revision: D23273579 fbshipit-source-id: 6f474f312014d41823689de2f7736b3563ff2b3f 28 August 2020, 00:28:49 UTC
ec1aa0f Vedanuj Goswami 27 August 2020, 18:02:02 UTC [docs] FB internal documentation (#518) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/518 Reviewed By: apsdehal Differential Revision: D23365885 fbshipit-source-id: ffbbe65ee6e8e9ae4dd05eb054cdf38df945f5e1 27 August 2020, 18:03:44 UTC
f5d2a88 Vedanuj Goswami 27 August 2020, 17:54:20 UTC [feat] Open image features with PathManager (#468) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/468 - Adds ability to read features from different URIs (Manifold, Everstore URIs) in addition to file paths. - Adds target for lmdb conversion/extraction Reviewed By: apsdehal Differential Revision: D23021466 fbshipit-source-id: 01cc4811730a02c98b57896e122e4a044f4546a7 27 August 2020, 17:55:55 UTC
78c5bfa Vedanuj Goswami 27 August 2020, 03:57:34 UTC [docs] Update BUTD documentation (#516) Summary: Update BUTD documentation - Move to website docs - Add more instructions about inference - Added table with updated best model Pull Request resolved: https://github.com/facebookresearch/mmf/pull/516 Test Plan: NA Reviewed By: apsdehal Differential Revision: D23351839 Pulled By: vedanuj fbshipit-source-id: 58a79b4bea6484f10e93425dc6032a4163082461 27 August 2020, 03:58:38 UTC
cf12af6 Vedanuj Goswami 27 August 2020, 03:49:24 UTC [fix] Freeze CircleCI black version (#517) Summary: Freeze black and flake8 versions in CircleCI to be in sync with the pre-commit hooks Pull Request resolved: https://github.com/facebookresearch/mmf/pull/517 Test Plan: Pass CircleCI tests Reviewed By: apsdehal Differential Revision: D23359588 Pulled By: vedanuj fbshipit-source-id: bdf7eb75b26ef26996bbb5ebb207841d099a04b6 27 August 2020, 03:50:37 UTC
31f0cc1 Vedanuj Goswami 27 August 2020, 02:50:05 UTC [feat] Generalized MM transformer (#422) Summary: Introduces a generalized version of multimodal transformer. - Extends `BaseTransformer` and builds a modular multimodal transformer model - Base transformer model can be any model from [transformers](https://github.com/huggingface/transformers) - Supports n number of modalities - Adds ability to add separate input, position and segment embeddings for each modality - text modalities share weight with base transformer model embeddings - position embeddings use separate embeddings for each modality - token type/segment embeddings are shared/mean of base transformer token/segment embeddings (if available) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/422 Test Plan: - Tested with HM, VQA2, OKVQA Reviewed By: apsdehal Differential Revision: D22636744 Pulled By: vedanuj fbshipit-source-id: 3fa05472f3486a62fb6d3a1ffcd689712af076b4 27 August 2020, 02:51:25 UTC
2db5ad3 Ronghang Hu 27 August 2020, 02:44:52 UTC [fix] load lr scheduler state dict during checkpoint loading (#513) Summary: the current implementation does not load lr_scheduler's states from checkpoints, which is a bug (causing last_epoch to be reset when loading from checkpoints). This commit fixes it by saving and loading lr_scheduler's states. It fall backs to setting lr_scheduler's last_epoch when lr_scheduler's states don't exist in checkpoint. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/513 Test Plan: tested with M4C checkpoint saving and loading. (CPU tests failed due to black formatting error in mmf/datasets/processors/bert_processors.py, unrelated to this PR.) Reviewed By: apsdehal Differential Revision: D23349944 Pulled By: ronghanghu fbshipit-source-id: 5c0802957804d3e169f30f0b944aa3d54ccf956a 27 August 2020, 02:46:02 UTC
aab972e Eric Undersander 27 August 2020, 02:39:13 UTC [feature, fix] added support for update_frequency (v2) (#367) Summary: This PR adds support for config.training.update_frequency: ``` # Run update_frequency=K batches with batch_size=N, accumulate gradients, then do # one update (one optimizer step). The effect is a large effective batch size of # KxN (without incurring the memory overhead of setting batch_size to KxN). update_frequency: 1 ``` https://www.internalfb.com/intern/tasks/?t=66569918 This is a revised approach; see also earlier PR https://github.com/facebookresearch/mmf/pull/292. With this change, we have to be more clear with naming. Amanpreet and I agreed on this: - we have no particular term for a single forward/backward pass for a single batch - "update" == "iteration" == one optimizer step (will be more than one batch when update_frequency > 1) The update logic in `BaseTrainer._backward` has mostly been moved to `_start_update` and `_finish_update`. There's also a fix for `MultiDatasetLoader.__len__`. The existing bug is only present for the case that the train set size is not a multiple of per-worker batch size, which is probably rare in practice. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/367 Test Plan: I did ad-hoc testing of BaseTrainer.train() (`mmf_run config="projects/hateful_memes/configs/late_fusion/defaults.yaml" model="late_fusion" dataset=hateful_memes`) with print statements and varying `[batch_size, update_frequency, train set size]`, including end-of-epoch and completion of the train run. Reviewed By: vedanuj Differential Revision: D23335921 Pulled By: apsdehal fbshipit-source-id: 984ae2706e50679eb0be47e14d872a359de36fd6 27 August 2020, 02:41:45 UTC
a5096b4 John Knox 26 August 2020, 15:40:54 UTC [mmf] Use static docs plugin Summary: I noticed mmf doesn't have the static docs plugin enabled. This will make sure your browser address bar and page titles work as expected. Reviewed By: apsdehal Differential Revision: D23343272 fbshipit-source-id: 1de615c72968722da162bf443d386301cbe60063 26 August 2020, 15:41:55 UTC
f700be8 John Knox 26 August 2020, 15:11:11 UTC [staticdocs] Update plugin version for all consumers Summary: Updates the static docs plugin everywhere. Done by running `./update-all-users.sh 0.3.6` inside `fbsource/xplat/staticdocs/` Reviewed By: mweststrate Differential Revision: D23343132 fbshipit-source-id: 2deaec1f8a4ead43475d77f40b7bf35e6b25f994 26 August 2020, 15:12:17 UTC
53ed195 Amanpreet Singh 25 August 2020, 23:07:18 UTC [fix] Unimodal base in case of batch size 2 (#502) Summary: Current implementation uses `len` to check whether output was from bert encoder or not. Add extra check for `torch.is_tensor` so that image features case for batch size 2 safely works. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/502 Test Plan: Tested on batch size 2. Reviewed By: vedanuj Differential Revision: D23243300 Pulled By: apsdehal fbshipit-source-id: 120469dc4ced94dd8348285223213d4409a3b22b 25 August 2020, 23:08:58 UTC
298718e Vedanuj Goswami 24 August 2020, 22:54:13 UTC [fix] Fix multilabel macro-f1 inheritance (#508) Summary: - Fixes MultiLabel Macro F1 metric. It should inherit from the `MultiLabelF1`. - Fixes an issue with a recent change in pytorch 1.7, where earlier argmax would return the index of ANY min or max, but now they will return the index of the FIRST min or max. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/508 Test Plan: Tests passed Reviewed By: apsdehal Differential Revision: D23283349 Pulled By: vedanuj fbshipit-source-id: e52dac18eed875767f251ef491329d1d5093bbf1 24 August 2020, 22:56:12 UTC
ab8d3f5 Vedanuj Goswami 22 August 2020, 01:50:19 UTC [feat] Add support for BS, NS metric calculation, update butd model etc. (#495) Summary: Fixes https://github.com/facebookresearch/mmf/issues/482 #489 Previously caption bleu4 metric in mmf could not be directly used with Beam Search(BS), Nucleus Sampling(NS). The process to follow was extract the predictions and then compute metrics separately. - Adds support for calculating metrics when using BS, NS - Uploaded new model to zoo trained with latest features. Old model when used with new features with mmf doesn't work. Issue https://github.com/facebookresearch/mmf/issues/489 - Modified `accumulate_tensor_fields` to throw a warning instead of KeyError when some metric field is not available. This is done to support use cases where different model output needs to be accumulated depending on train/val/test stages. - Fix zoo requirement for coco captioning - Fixes the issue of CUDA OOM when running inference with BS, NS on a GPU with less memory. Now only the captions(bs x cap_len) are used instead of scores (bs x cap_len x vocab_sz). Pull Request resolved: https://github.com/facebookresearch/mmf/pull/495 Test Plan: BLEU4 Score on val set : 36.04 ``` mmf_run config=projects/butd/configs/coco/beam_search.yaml run_type=val dataset=coco model=butd checkpoint.resume_zoo=butd ``` Reviewed By: apsdehal Differential Revision: D23214751 Pulled By: vedanuj fbshipit-source-id: 27f0cdc21cf09061d82a580440f4fa8ee13a1016 22 August 2020, 01:51:58 UTC
95e89ee Amanpreet Singh 20 August 2020, 15:56:35 UTC [feat] Support for more than 2 segments in MMBT (#460) Summary: Segment embedding other than first 2 dim are initialized from the average of first 2 embeddings This is required for the case where there are 3 segments such as post, ocr, and features. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/460 Reviewed By: vedanuj Differential Revision: D22970794 Pulled By: apsdehal fbshipit-source-id: 4d5ee86162079ae96d5387b0c08e96d73bb3bbe1 20 August 2020, 15:57:56 UTC
697ca3b Suzanna Sia 20 August 2020, 04:17:32 UTC [fix] Support for reading raw COCO images for VQA2 and COCO to support models like MMBT (#459) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/459 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/127 VQA2 and COCO dataset readers `dataset.py` previously only support reading in from feature file. After this PR, it should now be compatible with a model which reads in raw images. Should not affect any models which previously relied on `use_images: false`, `use_features: true` Changes: * Loads images in `{vqa2,coco}/{masked_,}dataset.py` if `self._use_features` flag is False * Included image transform in `faim/mmf/mmf/datasets/builders/vqa2/dataset.py:def init_processors(self)` * Enabled npy loading for internal `faim/mmf/mmf/datasets/databases/annotation_database.py:def _load_npy()` * Configs -- The main difference for configs is the inclusion of image_processor in this config file: `mmf/configs/datasets/vqa2/with_raw_images.yaml` and the dataset config `use_images` and `use_features` flags. ``` dataset_config: vqa2: use_images: true use_features: false images: train: - coco/defaults/images/train2014 val: - coco/defaults/images/val2014 test: - coco/defaults/images/test2015 ``` Reviewed By: apsdehal Differential Revision: D22979686 fbshipit-source-id: 3cac3d41329c098e0db1b50e4c554b72a194c120 20 August 2020, 04:18:37 UTC
16b20ff Amanpreet Singh 20 August 2020, 01:52:06 UTC [fix] Equal size proportional sampling and memory leak (#473) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/472 Here is the description that have been added to dataset loader on how sampling works after this change: Calculation of next batch is performed using following logic. Current chosen iterator is selected based on the dataset probabilities set in the `change_dataloader` function which is called every time `prepare_batch` is called. If we get the next batch from iterator without any StopIteration exception, we return it as it is. Otherwise, we have two cases: 1. In proportional sampling, since each dataset will have same number of epochs at any given time, we need to yield StopIteration exception when all iterators are finished. In turn, this will yield to `__iter__` all reignite all of the iterators. The code will not reach `__iter__` until unless all iterators are exhausted. 2. In the case of non-proportional sampling, epochs don't make sense. Think of a case of equal proportion sampling for dataset x and y where x is half the size of y. When x will complete its 2 epochs, y will have only 1 epoch completed. **So please don't use max_epochs or epoch based training in this case as it won't be honored**. If an iterator is finished, we just reignite it in this case and finished iterators variable isn't used. This means that this case will never reach the `__iter__` function ever again. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/473 Test Plan: Tests have been added to check size proportional sampling and equal sampling. Reviewed By: vedanuj Differential Revision: D23089370 Pulled By: apsdehal fbshipit-source-id: 06c09746624a32cb8bde5e52369d8891c34aa7e1 20 August 2020, 01:53:10 UTC
56e585d 在原佐为 19 August 2020, 18:20:11 UTC [fix] package configs in sdist package (#488) Summary: add config in MANIFEST.in to package yaml in mmf/configs. see https://github.com/facebookresearch/mmf/issues/479 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/488 Reviewed By: vedanuj Differential Revision: D23202899 Pulled By: apsdehal fbshipit-source-id: 6544e20e458c103912cdc9abbc4cf86d0b71ff73 19 August 2020, 18:21:40 UTC
6b94a8b Antonio V Mendoza 19 August 2020, 03:53:11 UTC [feat] Adding LXMERT to list of available VL models for finetuning and pretraining (#339) Summary: Adding model LXMERT. To pretrain on VQA run: `mmf_run config=projects/lxmert/configs/vqa2/pretrain.yaml run_type=train_val dataset=masked_vqa2 model=lxmert ` Added config files for GQA, COCO, VQA2.0, Visual Genome pretraining and VQA2 finetuning. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/339 Reviewed By: apsdehal Differential Revision: D22858282 Pulled By: vedanuj fbshipit-source-id: 09c8c04458df88866bf95ed6ed466042eb969900 19 August 2020, 03:54:16 UTC
962a7ea Suzanna Sia 19 August 2020, 00:43:08 UTC [fix] More informative logging for loading checkpoints. (#491) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/491 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/131 Previously unclear at what stage the loaded checkpoint file resumes training and this may have some unintended consequences. Adding `num_updates`, `current_iteration`, and `current_epoch` to the logger. Reviewed By: vedanuj Differential Revision: D23199211 fbshipit-source-id: c224d169f2aad681e235a3b820b2b824b5021055 19 August 2020, 00:44:25 UTC
da0b126 Vedanuj Goswami 19 August 2020, 00:12:58 UTC [fix] Set proper max_features for movie_mcan model (#492) Summary: Fixes https://github.com/facebookresearch/mmf/issues/484 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/492 Reviewed By: apsdehal Differential Revision: D23202564 Pulled By: vedanuj fbshipit-source-id: 459804792cc8bfc674ec06b0fb28fa973343b571 19 August 2020, 00:14:17 UTC
7b968a7 Vedanuj Goswami 18 August 2020, 20:15:09 UTC [chores] Add targets for sweep scripts (#130) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/130 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/490 Sweep script changes Reviewed By: lichengunc Differential Revision: D23183455 fbshipit-source-id: fcd1ead78170b95f333ac3cd1419d12be916506b 18 August 2020, 20:16:43 UTC
1fa896f Ning Zhang 18 August 2020, 04:15:25 UTC [fix] Typo in fusions for image feature (#129) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/129 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/487 Change image_features_0 to image_feature_0. All other image tokens are named as image_feature_0. Reviewed By: apsdehal Differential Revision: D23178741 fbshipit-source-id: ed26d445e4d3af5e7afa39f561cdaad14e80071f 18 August 2020, 04:16:48 UTC
9d76995 Amanpreet Singh 18 August 2020, 02:28:15 UTC [fix] Remove id from required params (#486) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/485 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/486 Reviewed By: vedanuj Differential Revision: D23181522 Pulled By: apsdehal fbshipit-source-id: 7efb7db5a6868e3c8d943e74cfc8f2f551168145 18 August 2020, 02:29:19 UTC
2811c1e Amanpreet Singh 14 August 2020, 21:45:45 UTC [refactor] Mimic formatter settings same as fbcode (#474) Summary: - Removes seed-isort-config - Seeds all third_party once, in future authors should manually update this - Add MMF also to third party to be comparable with internal - Pin black and isort versions same as internal - Update all files Pull Request resolved: https://github.com/facebookresearch/mmf/pull/474 Reviewed By: vedanuj Differential Revision: D23092121 Pulled By: apsdehal fbshipit-source-id: 9b7420408dca7ab588dc94b5d886073dcdad89c9 14 August 2020, 21:47:50 UTC
fda78ad Vedanuj Goswami 13 August 2020, 05:05:37 UTC [feat] Padded grid features (#464) Summary: Fixes issue https://github.com/facebookresearch/mmf/issues/453 . - Add support for variable feature size for grid based features. - This will enable using features from https://github.com/facebookresearch/grid-feats-vqa repository Pull Request resolved: https://github.com/facebookresearch/mmf/pull/464 Test Plan: Tested Movie+Mcan model with variable sized features. Reviewed By: apsdehal Differential Revision: D23054193 Pulled By: vedanuj fbshipit-source-id: 8fd1a96671e6bca0d2763398056014f20a8da13c 13 August 2020, 05:06:42 UTC
21af61a Vedanuj Goswami 13 August 2020, 00:13:09 UTC [mmf][feat] Allow option to override concat in multi sentence tokenizer (#471) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/471 Adding a config param for concatenating tokenizer output for multi sentence tokenizer. This is required for models which build separate embeddings for each sentence/modality in a sample. Reviewed By: apsdehal Differential Revision: D23070178 fbshipit-source-id: 2e3b3232c564d5eb46c304af6760348d98210966 13 August 2020, 00:14:36 UTC
918092f Yingxin Kang 12 August 2020, 19:00:44 UTC [feat] Install glove only when the processor is called (#469) Summary: Embeddings download happens in Vocab(..), which was previously called in __init__. We put it in __call__ instead to delay the download. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/469 Reviewed By: vedanuj, apsdehal Differential Revision: D23046828 Pulled By: Miiira fbshipit-source-id: 061d238ead0aa9dd3b10ff9cd0fe0abe7c74215a 12 August 2020, 19:02:03 UTC
6ac815d Suzanna Sia 12 August 2020, 08:39:45 UTC [fix] Allow user to define 'text_b' for bert_tokenizer text processor. (#128) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/128 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/461 Previously `bert_tokenizer` text processor assumed there are no `text_b` tokens. We still preserve this default logic (tokens_b defaults to None), but additionally allow the user to specify `text_b` if desired. Reviewed By: apsdehal Differential Revision: D22988310 fbshipit-source-id: 4d918380a59a4112483c95ec5cbdfd3641240dcf 12 August 2020, 08:40:52 UTC
1164df9 dependabot[bot] 11 August 2020, 19:47:55 UTC Bump prismjs from 1.20.0 to 1.21.0 in /website (#463) Summary: Bumps [prismjs](https://github.com/PrismJS/prism) from 1.20.0 to 1.21.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/PrismJS/prism/releases">prismjs's releases</a>.</em></p> <blockquote> <h2>v1.21.0</h2> <p>Release 1.21.0</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PrismJS/prism/blob/master/CHANGELOG.md">prismjs's changelog</a>.</em></p> <blockquote> <h2>1.21.0 (2020-08-06)</h2> <h3>New components</h3> <ul> <li><strong>.ignore</strong> &amp; <strong>.gitignore</strong> &amp; <strong>.hgignore</strong> &amp; <strong>.npmignore</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2481">#2481</a>) <a href="https://github.com/PrismJS/prism/commit/3fcce6fe"><code>3fcce6fe</code></a></li> <li><strong>Agda</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2430">#2430</a>) <a href="https://github.com/PrismJS/prism/commit/3a127c7d"><code>3a127c7d</code></a></li> <li><strong>AL</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2300">#2300</a>) <a href="https://github.com/PrismJS/prism/commit/de21eb64"><code>de21eb64</code></a></li> <li><strong>Cypher</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2459">#2459</a>) <a href="https://github.com/PrismJS/prism/commit/398e2943"><code>398e2943</code></a></li> <li><strong>Dhall</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2473">#2473</a>) <a href="https://github.com/PrismJS/prism/commit/649e51e5"><code>649e51e5</code></a></li> <li><strong>EditorConfig</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2471">#2471</a>) <a href="https://github.com/PrismJS/prism/commit/ed8fff91"><code>ed8fff91</code></a></li> <li><strong>HLSL</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2318">#2318</a>) <a href="https://github.com/PrismJS/prism/commit/87a5c7ae"><code>87a5c7ae</code></a></li> <li><strong>JS stack trace</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2418">#2418</a>) <a href="https://github.com/PrismJS/prism/commit/ae0327b3"><code>ae0327b3</code></a></li> <li><strong>PeopleCode</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2302">#2302</a>) <a href="https://github.com/PrismJS/prism/commit/bd4d8165"><code>bd4d8165</code></a></li> <li><strong>PureBasic</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2369">#2369</a>) <a href="https://github.com/PrismJS/prism/commit/d0c1c70d"><code>d0c1c70d</code></a></li> <li><strong>Racket</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2315">#2315</a>) <a href="https://github.com/PrismJS/prism/commit/053016ef"><code>053016ef</code></a></li> <li><strong>Smali</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2419">#2419</a>) <a href="https://github.com/PrismJS/prism/commit/22eb5cad"><code>22eb5cad</code></a></li> <li><strong>Structured Text (IEC 61131-3)</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2311">#2311</a>) <a href="https://github.com/PrismJS/prism/commit/8704cdfb"><code>8704cdfb</code></a></li> <li><strong>UnrealScript</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2305">#2305</a>) <a href="https://github.com/PrismJS/prism/commit/1093ceb3"><code>1093ceb3</code></a></li> <li><strong>WarpScript</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2307">#2307</a>) <a href="https://github.com/PrismJS/prism/commit/cde5b0fa"><code>cde5b0fa</code></a></li> <li><strong>XML doc (.net)</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2340">#2340</a>) <a href="https://github.com/PrismJS/prism/commit/caec5e30"><code>caec5e30</code></a></li> <li><strong>YANG</strong> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2467">#2467</a>) <a href="https://github.com/PrismJS/prism/commit/ed1df1e1"><code>ed1df1e1</code></a></li> </ul> <h3>Updated components</h3> <ul> <li>Markup &amp; JSON: Added new aliases (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2390">#2390</a>) <a href="https://github.com/PrismJS/prism/commit/9782cfe6"><code>9782cfe6</code></a></li> <li>Fixed several cases of exponential backtracking (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2268">#2268</a>) <a href="https://github.com/PrismJS/prism/commit/7a554b5f"><code>7a554b5f</code></a></li> <li><strong>APL</strong> <ul> <li>Added <code>⍥</code> (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2409">#2409</a>) <a href="https://github.com/PrismJS/prism/commit/0255cb6a"><code>0255cb6a</code></a></li> </ul> </li> <li><strong>AutoHotkey</strong> <ul> <li>Added missing <code>format</code> built-in (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2450">#2450</a>) <a href="https://github.com/PrismJS/prism/commit/7c66cfc4"><code>7c66cfc4</code></a></li> <li>Improved comments and other improvements (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2412">#2412</a>) <a href="https://github.com/PrismJS/prism/commit/ddf3cc62"><code>ddf3cc62</code></a></li> <li>Added missing definitions (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2400">#2400</a>) <a href="https://github.com/PrismJS/prism/commit/4fe03676"><code>4fe03676</code></a></li> </ul> </li> <li><strong>Bash</strong> <ul> <li>Added <code>composer</code> command (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2298">#2298</a>) <a href="https://github.com/PrismJS/prism/commit/044dd271"><code>044dd271</code></a></li> </ul> </li> <li><strong>Batch</strong> <ul> <li>Fix escaped double quote (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2485">#2485</a>) <a href="https://github.com/PrismJS/prism/commit/f0f8210c"><code>f0f8210c</code></a></li> </ul> </li> <li><strong>C</strong> <ul> <li>Improved macros and expressions (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2440">#2440</a>) <a href="https://github.com/PrismJS/prism/commit/8a72fa6f"><code>8a72fa6f</code></a></li> <li>Improved macros (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2320">#2320</a>) <a href="https://github.com/PrismJS/prism/commit/fdcf7ed2"><code>fdcf7ed2</code></a></li> </ul> </li> <li><strong>C#</strong> <ul> <li>Improved pattern matching (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2411">#2411</a>) <a href="https://github.com/PrismJS/prism/commit/7f341fc1"><code>7f341fc1</code></a></li> <li>Fixed adjacent string interpolations (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2402">#2402</a>) <a href="https://github.com/PrismJS/prism/commit/2a2e79ed"><code>2a2e79ed</code></a></li> </ul> </li> <li><strong>C++</strong> <ul> <li>Added support for default comparison operator (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2426">#2426</a>) <a href="https://github.com/PrismJS/prism/commit/8e9d161c"><code>8e9d161c</code></a></li> <li>Improved class name detection (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2348">#2348</a>) <a href="https://github.com/PrismJS/prism/commit/e3fe9040"><code>e3fe9040</code></a></li> <li>Fixed <code>enum class</code> class names (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2342">#2342</a>) <a href="https://github.com/PrismJS/prism/commit/30b4e254"><code>30b4e254</code></a></li> </ul> </li> <li><strong>Content-Security-Policy</strong> <ul> <li>Fixed directives (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2461">#2461</a>) <a href="https://github.com/PrismJS/prism/commit/537a9e80"><code>537a9e80</code></a></li> </ul> </li> <li><strong>CSS</strong> <ul> <li>Improved url and added keywords (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2432">#2432</a>) <a href="https://github.com/PrismJS/prism/commit/964de5a1"><code>964de5a1</code></a></li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PrismJS/prism/commit/187c8a607ee70c7914682870156faa31ed01f001"><code>187c8a6</code></a> 1.21.0</li> <li><a href="https://github.com/PrismJS/prism/commit/bf4f323391d546d4b2e1a388f05c512a27491e9c"><code>bf4f323</code></a> Changelog for v1.21.0 (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2507">#2507</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/8bba4880202ef6bd7a1e379fe9aebe69dd75f7be"><code>8bba488</code></a> Previewers: Fixed XSS (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2506">#2506</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/158caf52343e59a66c2351ff1d83648efe871e33"><code>158caf5</code></a> JSON: Greedy comments (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2479">#2479</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/f0f8210c1a9745c064d49bfb985544c654986b24"><code>f0f8210</code></a> Batch: Fix escaped double quote (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2485">#2485</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/649e51e56250a81dc0b0c4f5b3a4ea23e1c21834"><code>649e51e</code></a> Added support for Dhall (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2473">#2473</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/453079bf96746e0c44f90cb7cd90fcae9a5f94cc"><code>453079b</code></a> Line Numbers: Fixed class name on website</li> <li><a href="https://github.com/PrismJS/prism/commit/a0efa40bde420ac3923947be611ebe6b3f077dce"><code>a0efa40</code></a> Fixed Treeview page (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2484">#2484</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/78161d607fe7d493831ff19759aac951330134a1"><code>78161d6</code></a> VB: Added VBA alias (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2469">#2469</a>)</li> <li><a href="https://github.com/PrismJS/prism/commit/ed1df1e1208401a8b84330ecc39689fa37d0e9f6"><code>ed1df1e</code></a> Added support for YANG (<a href="https://github-redirect.dependabot.com/PrismJS/prism/issues/2467">#2467</a>)</li> <li>Additional commits viewable in <a href="https://github.com/PrismJS/prism/compare/v1.20.0...v1.21.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=prismjs&package-manager=npm_and_yarn&previous-version=1.20.0&new-version=1.21.0)](https://help.github.com/articles/configuring-automated-security-fixes) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/463 Reviewed By: apsdehal Differential Revision: D23031582 Pulled By: vedanuj fbshipit-source-id: 18eaa0b3ccdc135f261d195d0e195f2b5f0e929d 11 August 2020, 19:50:04 UTC
67390cb Parth Suresh 10 August 2020, 18:00:27 UTC [feat] Added VQACPv2 Dataset Implementation (#448) Summary: Adds support for VQACPv2 dataset in MMF. Automatic downloads are only supported at image level. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/448 Test Plan: A config has been added to test on MMBT with Images. `mmf_run config=projects/mmbt/configs/vqacp_v2/with_images.yaml run_type=train_val dataset=vqacp_v2 model=mmbt training.batch_size=64 training.seed=1` Reviewed By: vedanuj Differential Revision: D22928390 Pulled By: apsdehal fbshipit-source-id: 7aaac7581b96ace47cbd99a8cbf7fbb852c0519b 10 August 2020, 18:02:18 UTC
080a330 Amanpreet Singh 07 August 2020, 06:42:46 UTC [feat] Processor Pipeline for internal batch processors (#462) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/462 Decouples the pipeline for feature and text processor into separate functions and then allow it to be extensible. Users can inherit this processor and define and override some particular parts. This will allow easy extensibility in future. Also adds MultiSentenceBertTokenizer into the mix. Reviewed By: vedanuj Differential Revision: D22961076 fbshipit-source-id: f08f8e76fb049b3e97a321a525c1c0c00cf0ad15 07 August 2020, 06:44:11 UTC
61f14ba Amanpreet Singh 06 August 2020, 23:48:07 UTC [feat] Synchronize device for batch based on model (#456) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/456 Purpose of this diff is to add `to_device` method which is a helper utility method to move sample list to a proper device. In case the user is using custom dataset and forgot to move it to proper device, this changes will make sure that the batch is automatically moved to proper device at multiple stages of a training loop including making sure the batch model is receiving is on same device as the model's device. The checks are added at the following stages: - Before forward in training loop - Before forward in prediction loop - Inside BaseModel's forward This diff also remove the warning about prepare_batch in the multi dataset loader as it is not needed anymore. Reviewed By: vedanuj Differential Revision: D22955817 fbshipit-source-id: 5cd69cff518480f75dd8e9eda14aca31e575c48b 06 August 2020, 23:49:30 UTC
5b60950 Amanpreet Singh 06 August 2020, 20:53:01 UTC [feat] Allow custom keys in metrics and some new metrics (#447) Summary: - Users can specify custom keys for the metrics, for example in case of recall@k you can specify recall@50 as key and then pass parameter k - Add binary_ap and recall@precisionK metrics - Fix MicroAP and MacroAP inheritance Pull Request resolved: https://github.com/facebookresearch/mmf/pull/447 Test Plan: Tests added and modified. Reviewed By: vedanuj Differential Revision: D22633312 Pulled By: apsdehal fbshipit-source-id: d1319906b28f32bd900088a62f21fa1e6bb5bc13 06 August 2020, 20:54:16 UTC
7d89369 Sasha Sheng 06 August 2020, 20:09:58 UTC [fix] raise value error rather than exception; fixed formating with black (#458) Summary: [fix] raise value error rather than exception; fixed formating with black - I apologize with the premature land, I kicked off transfering code before I saw the comments. - Test with ``pytest tests`` - This passes tests Question, would you be open to using git precommit hooks to run isort, black automatically? Pull Request resolved: https://github.com/facebookresearch/mmf/pull/458 Reviewed By: apsdehal Differential Revision: D22982028 Pulled By: ytsheng fbshipit-source-id: 9c07b0dec7ff3b4ac241a1796415b186bbec3a59 06 August 2020, 20:11:18 UTC
faf8edd Suzanna Sia 06 August 2020, 04:51:32 UTC [fix, refactor] Tuple output bug in visual_bert model (#457) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/457 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/126 [fix]: The first object returned should always be return `encoded_layers[0]` or `sequence_output`, or there will be a `tuple` error. [refactor]; Remove duplicate lines by better placing of the conditional statement `if self.output_attentions` . No actual logic changed. Reviewed By: vedanuj, apsdehal Differential Revision: D22970391 fbshipit-source-id: 9617c9b6b9ab7defe59dcad127f19a6f045b94be 06 August 2020, 04:52:34 UTC
6329d7c Sasha Sheng 06 August 2020, 04:38:58 UTC [feat] Support epoch based training (#449) Summary: Hi y'all, this is my first PR, so let me know how you feel about my approach. :) Some logging was only added for this test plan and therefore does not exist in this PR. Let me know if you find those logging helpful. I can always add them back. A couple contributions: * This adds support for max_epochs. Using num_updates to control training loop; * If you have both num_epochs and num_updates specified, we warn you, and favor num_epochs. * This supports fractional epoch. * I think I found an issue where we are training max_updates + 1 iterations, this PR also fixes it. When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=None\ training.max_epochs=None ``` ![Screen Shot 2020-08-04 at 11 53 39 AM](https://user-images.githubusercontent.com/1545487/89333073-4feb5180-d649-11ea-9d38-8f480bec0bb2.png) When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=10\ training.max_epochs=1 ``` we favor max_epochs over max_updates, added a warning message. ![Screen Shot 2020-08-04 at 11 58 25 AM](https://user-images.githubusercontent.com/1545487/89333543-f3d4fd00-d649-11ea-8711-59e8d678cef1.png) When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=None\ training.max_epochs=1 ``` ![Screen Shot 2020-08-04 at 1 52 54 AM](https://user-images.githubusercontent.com/1545487/89274076-4637fe80-d5f5-11ea-89cc-131f167beb38.png) When cmd is: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml \ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=10\ training.max_epochs=None ``` ![Screen Shot 2020-08-04 at 1 13 11 AM](https://user-images.githubusercontent.com/1545487/89270083-b04da500-d5ef-11ea-9c0e-26cbba18f3ca.png) To test fractional epochs: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml\ datasets=textvqa\ model=m4c\ run_type=train_val\ training.max_updates=40\ training.max_epochs=0.04 ``` <img width="1073" alt="Screen Shot 2020-08-04 at 9 59 22 AM" src="https://user-images.githubusercontent.com/1545487/89323554-bb79f280-d63a-11ea-96d4-1cdafc47b7e2.png"> To test: pytest tests/trainers/test_training_loop.py ![Screen Shot 2020-08-04 at 4 58 05 PM](https://user-images.githubusercontent.com/1545487/89356885-c8ff9e80-d673-11ea-9f5c-43e9a520e508.png) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/449 Reviewed By: apsdehal Differential Revision: D22963208 Pulled By: ytsheng fbshipit-source-id: cfe9786314287c3f7f31ce2b46d602568b5ffd4c 06 August 2020, 04:40:27 UTC
2dc099a Sasha Sheng 06 August 2020, 00:17:01 UTC [fix] run isort (#452) Summary: 76 passed, 1 skipped, 31 warnings in 91.18s Tests: `pytest tests` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/452 Reviewed By: apsdehal Differential Revision: D22959790 Pulled By: ytsheng fbshipit-source-id: 9e0480c147ccf6caff82c41ab6c41bbe9bba18e2 06 August 2020, 00:18:47 UTC
5e41e02 Sasha Sheng 04 August 2020, 19:59:45 UTC Fix black reformatting errors (#451) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/451 Reviewed By: vedanuj Differential Revision: D22927669 Pulled By: ytsheng fbshipit-source-id: c1fd5f5d45133004a1c2899d019daadb4d1fb3c8 04 August 2020, 20:01:52 UTC
6fd7853 Amanpreet Singh 03 August 2020, 18:38:28 UTC [fix] Captioning predictions for bsz 1, remove opinions in testreporter (#437) Summary: - Use object_to_byte and reverse for saving image ids - Remove opinions about shapes and gathering in the test reporter based on keys. Generalized logic now. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/437 Reviewed By: vedanuj Differential Revision: D22836831 Pulled By: apsdehal fbshipit-source-id: 4ff6b977c41b017e15b66b499fabf7883754c1ef 03 August 2020, 18:39:56 UTC
efa331e Charles Bai 03 August 2020, 18:17:30 UTC [fix] Override of final array elements through dot element (#438) Summary: Should check if we reach the end of the dot_list when overriding config with args. This should fix issue https://github.com/facebookresearch/mmf/issues/431 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/438 Test Plan: ``` mmf_run config=projects/m4c/configs/textvqa/defaults.yaml \ datasets=textvqa \ model=m4c \ run_type=train_val \ training.lr_steps[1]=100 \ dataset_config.textvqa.zoo_requirements[0]="test" \ ``` Output: ``` Overriding option config to projects/m4c/configs/textvqa/defaults.yaml Overriding option datasets to textvqa Overriding option model to m4c Overriding option run_type to train_val Overriding option training.lr_steps[1] to 100 Overriding option dataset_config.textvqa.zoo_requirements[0] to test ``` Also: `pytest ./tests/configs/test_config_overrides.py` Reviewed By: apsdehal Differential Revision: D22894342 Pulled By: charlesxbai fbshipit-source-id: bbd67b0c223c571bd2f705299d4a48cf48a49655 03 August 2020, 18:18:58 UTC
9ba7af6 Hu Xu 01 August 2020, 05:27:58 UTC [mmf][fix]Assertion message to match if condition Summary: Correction of assertion message on the first dimension . Reviewed By: apsdehal Differential Revision: D22860233 fbshipit-source-id: d8280d81a84ee23b51512f987302b4397318b0a5 01 August 2020, 05:29:00 UTC
7cdbf8c dependabot[bot] 31 July 2020, 22:25:45 UTC Bump elliptic from 6.5.2 to 6.5.3 in /website (#439) Summary: Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.2 to 6.5.3. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/indutny/elliptic/commit/8647803dc3d90506aa03021737f7b061ba959ae1"><code>8647803</code></a> 6.5.3</li> <li><a href="https://github.com/indutny/elliptic/commit/856fe4d99fe7b6200556e6400b3bf585b1721bec"><code>856fe4d</code></a> signature: prevent malleability and overflows</li> <li>See full diff in <a href="https://github.com/indutny/elliptic/compare/v6.5.2...v6.5.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=elliptic&package-manager=npm_and_yarn&previous-version=6.5.2&new-version=6.5.3)](https://help.github.com/articles/configuring-automated-security-fixes) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/439 Reviewed By: apsdehal Differential Revision: D22858238 Pulled By: vedanuj fbshipit-source-id: 040488c2037bc4235635ffffc102e341da7140db 31 July 2020, 22:27:24 UTC
cb2adc6 Amanpreet Singh 30 July 2020, 20:25:19 UTC [feat,fix] Logger overhaul, shift to Python logger (#436) Summary: - All logging uses python based logging after this - Colored logging support inspired from d2 - Wherever touched moved to format strings - Adds options for colored logging - Moves torch version and cuda devices to top - Moves all print and writer statements to logger - Also, fixes the long failing windows tests for logger What are the benefits of doing this? - All libraries which use python logging will receives our logs now and we receive theirs - Proper division between stderr and stdout Pull Request resolved: https://github.com/facebookresearch/mmf/pull/436 Test Plan: Tested locally Reviewer suggestions: This PR needs to be reviewed somewhat carefully to make sure I don't miss anything and as it touches all of the MMF. Reviewed By: vedanuj Differential Revision: D22819619 Pulled By: apsdehal fbshipit-source-id: 6675ab90567c0d192c3023ede1cdb183dd103ea6 30 July 2020, 20:26:48 UTC
3fc0cdf Vedanuj Goswami 28 July 2020, 01:45:11 UTC [feat] Add Movie+Mcan grid features to zoo (#428) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/428 Added the grid features used for movie+mcan model to dataset zoo for automated downloads. Reviewed By: apsdehal Differential Revision: D22721729 fbshipit-source-id: 51757819404510a48b984f5437ac1fc023c59e65 28 July 2020, 01:46:12 UTC
9f28973 Amanpreet Singh 27 July 2020, 20:54:07 UTC [fix] Move anomaly detection to an option (#434) Summary: - As per PyTorch docs anomaly detection should only be turned on for debugging purposes and can slow down things - This warning wasn't there when the code was initially added so we missed this. Setting this to false will drastically improve performance of all our models. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/434 Test Plan: Tested locally on VisualBERT with the setting off Reviewed By: vedanuj Differential Revision: D22768897 Pulled By: apsdehal fbshipit-source-id: f990406fc1b8e294aeb726013eeaef44c453b4c4 27 July 2020, 20:55:28 UTC
c75f245 Vedanuj Goswami 22 July 2020, 04:57:47 UTC [feat] Add Base transformer class for modular multimodal transformers (#398) Summary: - Adds a base transformer model class with modular components for constructing a transformer based multi modal model. The modularity is divided as : ``` self.build_encoders() self.build_embeddings() self.build_transformer() self.build_heads() self.build_losses() ``` - Base model automatically loads an encoder specified by `transformer_base` in config. For example it can be `bert-base-uncased`, `roberta-base` etc. - Ability to preprocess the input sample_list in `preprocess_sample ` according to model's requirement and also postprocess the output from the transformer according to requirement. For pretraining, we can add cls head and calculate loss on the output in the `postprocess_output ` method. For classification we can calculate scores etc. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/398 Test Plan: - Tested with creating a new model using the base class and running on Hateful Memes, VQA2 Reviewed By: apsdehal Differential Revision: D22480254 Pulled By: vedanuj fbshipit-source-id: 7eec160941f2146c8822d7590d50acb7c74b6904 22 July 2020, 04:58:55 UTC
18bcbd8 dependabot[bot] 22 July 2020, 04:20:32 UTC [chores] Bump lodash from 4.17.15 to 4.17.19 in /website (#423) Summary: Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/lodash/lodash/releases">lodash's releases</a>.</em></p> <blockquote> <h2>4.17.16</h2> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/lodash/lodash/commit/d7fbc52ee0466a6d248f047b5d5c3e6d1e099056"><code>d7fbc52</code></a> Bump to v4.17.19</li> <li><a href="https://github.com/lodash/lodash/commit/2e1c0f22f425e9c013815b2cd7c2ebd51f49a8d6"><code>2e1c0f2</code></a> Add npm-package</li> <li><a href="https://github.com/lodash/lodash/commit/1b6c282299f4e0271f932b466c67f0f822aa308e"><code>1b6c282</code></a> Bump to v4.17.18</li> <li><a href="https://github.com/lodash/lodash/commit/a370ac81408de2da77a82b3c4b61a01a3b9c2fac"><code>a370ac8</code></a> Bump to v4.17.17</li> <li><a href="https://github.com/lodash/lodash/commit/1144918f3578a84fcc4986da9b806e63a6175cbb"><code>1144918</code></a> Rebuild lodash and docs</li> <li><a href="https://github.com/lodash/lodash/commit/3a3b0fd339c2109563f7e8167dc95265ed82ef3e"><code>3a3b0fd</code></a> Bump to v4.17.16</li> <li><a href="https://github.com/lodash/lodash/commit/c84fe82760fb2d3e03a63379b297a1cc1a2fce12"><code>c84fe82</code></a> fix(zipObjectDeep): prototype pollution (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4759">#4759</a>)</li> <li><a href="https://github.com/lodash/lodash/commit/e7b28ea6cb17b4ca021e7c9d66218c8c89782f32"><code>e7b28ea</code></a> Sanitize sourceURL so it cannot affect evaled code (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4518">#4518</a>)</li> <li><a href="https://github.com/lodash/lodash/commit/0cec225778d4ac26c2bac95031ecc92a94f08bbb"><code>0cec225</code></a> Fix lodash.isEqual for circular references (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4320">#4320</a>) (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4515">#4515</a>)</li> <li><a href="https://github.com/lodash/lodash/commit/94c3a8133cb4fcdb50db72b4fd14dd884b195cd5"><code>94c3a81</code></a> Document matches* shorthands for over* methods (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4510">#4510</a>) (<a href="https://github-redirect.dependabot.com/lodash/lodash/issues/4514">#4514</a>)</li> <li>Additional commits viewable in <a href="https://github.com/lodash/lodash/compare/4.17.15...4.17.19">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~mathias">mathias</a>, a new releaser for lodash since your current version.</p> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=lodash&package-manager=npm_and_yarn&previous-version=4.17.15&new-version=4.17.19)](https://help.github.com/articles/configuring-automated-security-fixes) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/423 Reviewed By: vedanuj Differential Revision: D22657049 Pulled By: apsdehal fbshipit-source-id: 924e15b18c730b2ddbd3fb12b431536463d8bff4 22 July 2020, 04:21:30 UTC
9c2662f Vedanuj Goswami 22 July 2020, 03:29:19 UTC [fix] Rename transformer_bbox_processor as bbox_processor already exists (#424) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/424 Fix `bbox_processor` bug due to duplicate name of processors. Rename to `transformer_bbox_processor` for transformer specific bbox processor Reviewed By: apsdehal Differential Revision: D22655123 fbshipit-source-id: d0c8e070afaf726086c1d34fd4040b0eacaf8615 22 July 2020, 03:30:37 UTC
3d0c2ae Amanpreet Singh 22 July 2020, 02:37:43 UTC [feat] OKVQA dataset implementation (#420) Summary: Adds support for OKVQA dataset in MMF. Automatic downloads are only supported at image level, feature support maybe added later. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/420 Test Plan: A config has been added to test on MMBT with Images. ``` mmf_run config=projects/mmbt/configs/okvqa/with_images.yaml run_type=train_val dataset=okvqa model=mmbt training.batch_size=64 training.stdout_capture=False training.seed=1 ``` Reviewed By: vedanuj Differential Revision: D22608435 Pulled By: apsdehal fbshipit-source-id: 775d19d93063ce3b8cbcb875e17ce4ef2dc2f2f7 22 July 2020, 02:38:52 UTC
4398068 Stanislau Hlebik 18 July 2020, 00:07:23 UTC remediation of S205607 fbshipit-source-id: 798decc90db4f13770e97cdce3c0df7d5421b2a3 18 July 2020, 00:18:47 UTC
a7ba97f Stanislau Hlebik 18 July 2020, 00:07:23 UTC remediation of S205607 fbshipit-source-id: 5113fe0c527595e4227ff827253b7414abbdf7ac 18 July 2020, 00:18:47 UTC
c4d9f0b Vedanuj Goswami 17 July 2020, 22:17:44 UTC [WIP][fix] Run actions on PRs as well (#419) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/419 Reviewed By: apsdehal Differential Revision: D22603770 Pulled By: vedanuj fbshipit-source-id: 774b42a1dfe6209844d92ecc6ab621c4463b9e06 17 July 2020, 22:18:50 UTC
e11e2b5 Vedanuj Goswami 17 July 2020, 05:53:48 UTC [fix] Fix windows logger tests, HM Readme (#418) Summary: Fixes https://github.com/facebookresearch/mmf/issues/417 , Fixes windows test error due to permissions removing temp dir. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/418 Reviewed By: apsdehal Differential Revision: D22592532 Pulled By: vedanuj fbshipit-source-id: 28e0e80ad52556716279c5334acdffddf0340cc8 17 July 2020, 05:54:55 UTC
466155e Amanpreet Singh 17 July 2020, 01:58:44 UTC [fix] Checkpoint resume dir and value in sweeps scripts (#412) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/412 - Checkpoint resume wasn't enabled in fblearner script. - Checkpoint.resume should directly checked truthy. This will allow users to pass `checkpoint.resume=1` and avoid confusion. Reviewed By: vedanuj Differential Revision: D22566067 fbshipit-source-id: 18975236769c7cb554b7f2e28c7cc8d6a8b82ceb 17 July 2020, 01:59:56 UTC
1e1edce Amanpreet Singh 17 July 2020, 01:40:39 UTC [feat] FB Defaults (#413) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/413 Separate out fb defaults into a separate config for future use. Reviewed By: vedanuj Differential Revision: D22566143 fbshipit-source-id: b21b697f1ef9686127200a94f96d6db7b464869e 17 July 2020, 01:41:47 UTC
93c0300 Vedanuj Goswami 17 July 2020, 00:16:54 UTC [refactor] Add transformer bbox processor (#397) Summary: - Refactor `bbox` position processing for transformer based models - Fix feature path in masked coco dataset Pull Request resolved: https://github.com/facebookresearch/mmf/pull/397 Test Plan: - Test with VilBERT model on VQA2, Masked COCO Reviewed By: apsdehal Differential Revision: D22479935 Pulled By: vedanuj fbshipit-source-id: aa310d1d963facd058ba6a5322031514b15cb01c 17 July 2020, 00:18:26 UTC
d028012 Amanpreet Singh 16 July 2020, 22:13:15 UTC [feat] Add support for BatchProcessors and identity feat encoder (#411) Summary: - BatchProcessor type to initialize batche based processors - Also add support for identity feature encoder Pull Request resolved: https://github.com/facebookresearch/mmf/pull/411 Reviewed By: vedanuj Differential Revision: D22565866 Pulled By: apsdehal fbshipit-source-id: 39fb8af1bf118d0c92eb2e7c7609140def0e593d 16 July 2020, 22:14:23 UTC
f2ee179 Amanpreet Singh 16 July 2020, 20:05:54 UTC [refactor] Release some restrictions on supported dataset types (#410) Summary: - Allows datasets which are not inherited from BaseDataset to be returned from the DatasetBuilder - This will allow us to use IterableDataset which is required for some internal dataset - Batches can be directly returned now - Sampler doesn't need to have set_epoch method - Move fast read to VQABuilder as that's the only builder using it currently. - If prepare_batch is not implemented, throw a warning instead of error. User needs to implement it themselves now - `update_registry_for_model` is optional as well. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/410 Reviewed By: vedanuj Differential Revision: D22563932 Pulled By: apsdehal fbshipit-source-id: df68a8b330fcfd598d0428d848e90084f67965bb 16 July 2020, 20:07:42 UTC
4ba79d5 Amanpreet Singh 16 July 2020, 17:34:57 UTC [fix] COCO dataset, checksum and missing annotation dbs (#416) Summary: - Also, fixes https://github.com/facebookresearch/mmf/issues/415 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/416 Test Plan: - Tested BUTD on COCO Reviewed By: vedanuj Differential Revision: D22573643 Pulled By: apsdehal fbshipit-source-id: 1758e60ab4263e88438c748b7275685b98e3100e 16 July 2020, 17:36:33 UTC
e1adfc4 Shir Gur 16 July 2020, 16:51:36 UTC [fix] Fix image_database (#403) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/403 - Removes adding ".XXX" extensions in ```_get_attrs``` - Change ```from_path``` to use full image path Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/123 Reviewed By: apsdehal Differential Revision: D22473774 Pulled By: shirgur fbshipit-source-id: 876ea92d72527f89eabdc4f925cb9408216bef36 16 July 2020, 16:53:02 UTC
a79bb99 Vedanuj Goswami 16 July 2020, 02:06:41 UTC [feat] Add support for manifold image databases and exp save directory (#407) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/407 - Adds support for manifold image database - Adds manifold handler - Fix checkpoint and logger to support creating save_dir in manifold - Created new bucket for FAIR MMF : `manifold://fair_mmf/tree/data` Reviewed By: apsdehal Differential Revision: D22544389 fbshipit-source-id: e5cfc660b3e815d1495f1968dfb8695d2b3c58ea 16 July 2020, 02:08:19 UTC
42019cc Amanpreet Singh 15 July 2020, 19:46:48 UTC [fix] Isort version in linux as well (#408) Summary: - Also, prints the version of the linters used now Pull Request resolved: https://github.com/facebookresearch/mmf/pull/408 Test Plan: To be tested on CI Reviewed By: vedanuj Differential Revision: D22551651 Pulled By: apsdehal fbshipit-source-id: 4d5d5cb44c9720de60c2d395c102bc55346b2574 15 July 2020, 19:48:16 UTC
11174ad Amanpreet Singh 15 July 2020, 05:53:18 UTC [fix] Isort version same as pre-commit to avoid CI failures (#406) Summary: - With recent update of isort, it is changing alias imports to custom format, which is breaking our CI builds - Later, we should look into how we can move to the latest version of isort Pull Request resolved: https://github.com/facebookresearch/mmf/pull/406 Test Plan: To be tested on CI Reviewed By: vedanuj Differential Revision: D22542173 Pulled By: apsdehal fbshipit-source-id: 35e5cea04e8f6d61ac37f6c1482699dda9205c12 15 July 2020, 05:54:30 UTC
4a9862b Ivan Petrov Evtimov 14 July 2020, 02:29:08 UTC Checkpoint mmf training only last n intervals (#401) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/401 Per task 67509713, modify mmf model training to checkpoint only last n checkpoints Reviewed By: apsdehal Differential Revision: D22406213 fbshipit-source-id: 130fb5af458fce44dada8b28846665b039085e00 14 July 2020, 02:30:21 UTC
6fd6cb9 Amanpreet Singh 13 July 2020, 23:13:05 UTC [feat] Support internal website for MMF (#400) Summary: - Adds package dependency on MMF internal website docusaurus package Pull Request resolved: https://github.com/facebookresearch/mmf/pull/400 Test Plan: Tested locally See relevant diff: D22510353 Reviewed By: vedanuj Differential Revision: D22510683 Pulled By: apsdehal fbshipit-source-id: f7634b7579124610702647ea68b89d6ac8b38f13 13 July 2020, 23:14:10 UTC
4ef79b0 Amanpreet Singh 09 July 2020, 21:19:45 UTC [fix] Distributed seed should be set different for each process (#394) Summary: - This causes issues for things which should be random in different processes instead of being same like RandomSampler Pull Request resolved: https://github.com/facebookresearch/mmf/pull/394 Test Plan: - Tested on VisualBERT with VQA2 cc ronghanghu Reviewed By: vedanuj Differential Revision: D22450741 Pulled By: apsdehal fbshipit-source-id: b8d86e98840be7e2e7d7893930f1e6fb3406c3a7 09 July 2020, 21:20:53 UTC
f5fc894 Amanpreet Singh 08 July 2020, 22:22:40 UTC [refactor] Make build method public in annotation db and mmf dataset (#393) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/393 Test Plan: Tested on m4c Reviewed By: vedanuj Differential Revision: D22438277 Pulled By: apsdehal fbshipit-source-id: 1ce2af376f62d6886346e1ed05ef0dac145bb83b 08 July 2020, 22:25:14 UTC
948f6fe Shir Gur 07 July 2020, 07:11:41 UTC [feat] Add pre/post processing for predictions (#122) Summary: Adding ```on_prediction_start```and ```on_prediction_end``` for pre/post processing of the predictions Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/122 Reviewed By: vedanuj, apsdehal Differential Revision: D22397973 Pulled By: shirgur fbshipit-source-id: 957b364b393c92ec90f1b50c12a827bbfd3c7bc1 07 July 2020, 07:13:03 UTC
b77e8b2 Amanpreet Singh 07 July 2020, 05:28:24 UTC [tests] Add test for checking if shasum is valid for zoo (#380) Summary: - Uses a regex to make sure all of the checksums are valid sha256sums in the zoo configs - Also refactor test for zoos to be reusable Pull Request resolved: https://github.com/facebookresearch/mmf/pull/380 Reviewed By: vedanuj Differential Revision: D22406875 Pulled By: apsdehal fbshipit-source-id: 063e222372fcf03d6d67a2273f6439c1a5db7fd5 07 July 2020, 05:29:42 UTC
08001d3 Amanpreet Singh 07 July 2020, 04:21:32 UTC [fix] Processor id in sidebar and deploy (#391) Summary: - Deploy is broken due to wrong id in the processor docs Pull Request resolved: https://github.com/facebookresearch/mmf/pull/391 Reviewed By: vedanuj Differential Revision: D22406872 Pulled By: apsdehal fbshipit-source-id: 1006495e90d1481cba096aed0b0b88168b0e0367 07 July 2020, 04:23:53 UTC
31e6d7b Suzanna Sia 07 July 2020, 02:02:42 UTC [fix, refactor] Truncate bug and readable convert to index (#383) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/383 Fix a bug in `truncate_seq_pair` where if `tokens_b` is not `None`, the `max_length` should be `max_length-3` because there are 3 special tokens, [CLS] tokens_a [SEP] tokens_b [SEP]. Refactor `tokens` and `segment_ids` in the same sequential form as `lm_label_ids` for readability, does not change the code logic. Reviewed By: apsdehal Differential Revision: D22344634 fbshipit-source-id: 7aec985843e225587df74c3b4abe587abeb7f394 07 July 2020, 02:04:15 UTC
3d78fb7 Suzanna Sia 07 July 2020, 01:47:05 UTC [refactor] Gathering of distributed tensors based on number of dimensions (#384) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/384 Refactor to automatically check the dimensions and apply the appropriate reshaping before gathering distributed tensors. To make writing custom reports more convenient when new keys are added. � Did not change any of the actual logic. Makes the reshaping explicit depending on the number of the dimensions. Gathering distributed tensors increases the number of dimensions by 1 and we reshape accordingly. Will fail more gracefully if the tensor shape before `gather_tensor` is 0 or greater than 2. Reviewed By: vedanuj, apsdehal Differential Revision: D22299088 fbshipit-source-id: dbe71227a147bacf426fe3b6cc5c7eea463f8c7e 07 July 2020, 01:48:01 UTC
4a5c707 Amanpreet Singh 06 July 2020, 23:15:42 UTC [fix] Epoch counting in trainer (#389) Summary: - Current epoch counting has been broken and overestimates the count Pull Request resolved: https://github.com/facebookresearch/mmf/pull/389 Test Plan: Test on m4c/textvqa in distributed settings as well Reviewed By: vedanuj Differential Revision: D22399463 Pulled By: apsdehal fbshipit-source-id: 7714bb91cce5581fffcfadbc464325d9a4856517 06 July 2020, 23:16:55 UTC
2dc3190 Vedanuj Goswami 06 July 2020, 19:40:37 UTC [docs] Fix MoViE+MCAN docs (#388) Summary: Some doc fixes Pull Request resolved: https://github.com/facebookresearch/mmf/pull/388 Reviewed By: apsdehal Differential Revision: D22393314 Pulled By: vedanuj fbshipit-source-id: 5f91044a3093177074943a92f85b8466857205a4 06 July 2020, 19:42:16 UTC
c91d660 Amanpreet Singh 06 July 2020, 17:04:03 UTC [fix] Call load_state_dict in on_init_start and call it in trainer.load (#385) Summary: - This fixes the issue with loading distributed or data parallel models using checkpoint as model would parallelize but still won't have checkpoint loaded - A missing call to on_init_start has also been added Pull Request resolved: https://github.com/facebookresearch/mmf/pull/385 Test Plan: Tested by Omkar Reviewed By: vedanuj Differential Revision: D22370051 Pulled By: apsdehal fbshipit-source-id: 2176d4f410f9313a9bcf18ac5a0eec9338df2fc2 06 July 2020, 17:05:16 UTC
47da75a Amanpreet Singh 03 July 2020, 03:18:20 UTC [feat,fix] New textvqa features, retrained model, some fixes (#375) Summary: - Adds new features for TextVQA based on maskrcnn benchmark script which now also contains extra info as required to fix https://github.com/facebookresearch/mmf/issues/328 - Adds retrained model using these new features on VMB, on test-std this model gets 41.16% higher than the previous model - Fixes the issue with the answer processor and in general predictions on TextVQA for multiple models (LoRRA and M4C) - Fixes joint_stvqa script to properly load features for STVQA. The fix is hacky as of now and requires some refactoring in datasets to properly support dataset class attributes - Fixes the broken extract features scripts to use latest code and functions as well downlaod functionalities Pull Request resolved: https://github.com/facebookresearch/mmf/pull/375 Test Plan: Tested extensively on TextVQA with LoRRA as well as M4C Reviewed By: ronghanghu Differential Revision: D22321533 Pulled By: apsdehal fbshipit-source-id: fa97a7d56a333c6e1b6ffb1103e69cbaf7b1d783 03 July 2020, 03:19:36 UTC
b7287eb Amanpreet Singh 02 July 2020, 19:05:46 UTC [fix] Correct reference of tensorboard argument to clip gradient (#382) Summary: - This was causing the training to fail after recent trainer refactor in https://github.com/facebookresearch/mmf/issues/332 - Will properly fix https://github.com/facebookresearch/mmf/issues/378 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/382 Reviewed By: vedanuj, ronghanghu Differential Revision: D22353205 Pulled By: apsdehal fbshipit-source-id: 8f719542598b091b346496f702ca1d898ecf7d02 02 July 2020, 19:07:17 UTC
95c6287 Vedanuj Goswami 01 July 2020, 18:14:51 UTC [fix] Fix should_log in training_loop (#379) Summary: Fixes https://github.com/facebookresearch/mmf/issues/378 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/379 Reviewed By: apsdehal Differential Revision: D22334049 Pulled By: vedanuj fbshipit-source-id: faa51b4c8cf865850851df334b9971de92b641d1 01 July 2020, 18:16:08 UTC
b823f25 Amanpreet Singh 01 July 2020, 17:59:27 UTC [fix] VisualBERT regression of pooler; provide pooler strategy option (#377) Summary: - In VQA2 and general question answering, original codebase has a different pooler strategy of taking hidden state from second last token - The regression was causing issue with mismatch of accuracies on pretraining it right paper's checkpoint Pull Request resolved: https://github.com/facebookresearch/mmf/pull/377 Test Plan: Testing on the best checkpoint and estimated the accuracy to be same. Reviewed By: vedanuj Differential Revision: D22328981 Pulled By: apsdehal fbshipit-source-id: bcd289181289e500c29d3fd73e1a8600f0ce5b3d 01 July 2020, 18:00:37 UTC
d1d10cb Vedanuj Goswami 01 July 2020, 07:07:15 UTC [feat] Add MoViE+MCAN model (#345) Summary: - Adding the MoVie+MCAN model (VQA 2020 Challenge Winner) TODOs: - [x] Add typings to movie_mcan.py - [x] Upload the annotations to zoo - [x] Upload best model to zoo - [x] Cleanup feature_readers.py - [x] Update Readme Pull Request resolved: https://github.com/facebookresearch/mmf/pull/345 Reviewed By: apsdehal Differential Revision: D22214947 Pulled By: vedanuj fbshipit-source-id: 99da94efdbf34b74338b62299570a299860cb6d1 01 July 2020, 07:08:33 UTC
a1a1a55 Amanpreet Singh 01 July 2020, 06:41:20 UTC [fix] Reloads in spawned processes shouldn't setup imports again (#374) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/355 partially - In general, only setup_imports in the main processor - This means that the child processes are using only shallow registry which is fine as setup imports can cause issues in file not related with syntax error - Overall, this can't be fixed completely as by the nature of the spawn, the child will still reload all of the code and if a syntax error is there it will still break the worker. General suggestion is to keep two copies of the code, one for development and one for running and keep them in sync via GitHub Pull Request resolved: https://github.com/facebookresearch/mmf/pull/374 Test Plan: Tested with command and reproduction steps provided by Ronghang Reviewed By: vedanuj Differential Revision: D22322313 Pulled By: apsdehal fbshipit-source-id: 6a8e388df1244ff450f726bc66535ff53900da14 01 July 2020, 06:42:39 UTC
208579e Amanpreet Singh 30 June 2020, 22:17:38 UTC [docs] Tutorial on SLURM sweeps (#359) Summary: - Runs prettier on rest of docs as well - Point original readme to the new doc Pull Request resolved: https://github.com/facebookresearch/mmf/pull/359 Reviewed By: vedanuj Differential Revision: D22310372 Pulled By: apsdehal fbshipit-source-id: 4b3e61097fba319d8ea05c7e7d2cfda3d7074593 30 June 2020, 22:18:54 UTC
f021a7a Amanpreet Singh 30 June 2020, 21:53:03 UTC [fix] Resolve config when it is created (#371) Summary: - Fixes https://github.com/facebookresearch/mmf/issues/370. See it for details Pull Request resolved: https://github.com/facebookresearch/mmf/pull/371 Test Plan: Tested with visual bert run Reviewed By: vedanuj Differential Revision: D22297254 Pulled By: apsdehal fbshipit-source-id: 1a6549774dacf2f8cda492c9d2092b2ce42ccba7 30 June 2020, 21:54:11 UTC
025f05d Durgesh 30 June 2020, 16:28:20 UTC [fix] Installation Link in hateful_memes/README.md (#368) Summary: Broken Link in hateful_memes/README.md for mmf installation Fix Pull Request resolved: https://github.com/facebookresearch/mmf/pull/368 Reviewed By: apsdehal Differential Revision: D22291475 Pulled By: vedanuj fbshipit-source-id: 239ab1891576d234c8a146826e3999298474cc7a 30 June 2020, 16:29:19 UTC
b620c01 Vedanuj Goswami 30 June 2020, 16:25:26 UTC [fix] Rename imdb to annotation_db (#372) Summary: Renames remaining `imdb` to `annotation_db` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/372 Reviewed By: apsdehal Differential Revision: D22304951 Pulled By: vedanuj fbshipit-source-id: 47ead15b4741d1c0628782951cc3232992d444c7 30 June 2020, 16:26:28 UTC
48631d8 Vedanuj Goswami 30 June 2020, 03:57:09 UTC [feat, refactor] Trainer class refactor (#332) Summary: PR refactors the trainer class providing flexibility to extend from the BaseTrainer class and customize it. We also introduce a callback mechanism that can be executed at different events during training, validation or inference. Hence separates most of the logic of core training/optimization from other logistics. - Create an abstract base trainer class, and core functionalities like loading dataset, model, optimizer, metrics are abstract methods that has to be implemented by derived classes. - Create a new `MMFTrainer` class which keeps the same behavior as the current trainer but derived from the `BaseTrainer`. - Move core logic of training loop, evaluation loop and prediction loop to Mixin classes and can be easily used in trainer classes. - Create a callback mechanism similar to `PytorchLightning` with different types of events when callbacks can happen. With this it will be easier to have `PytorchLightning` supported trainer classes for `mmf` - Callbacks are executed before or after events like training, validation, test, batch, epoch etc. - Typing introduced in the trainer classes - There can be some additional refactoring around how to include some additional callbacks like Early Stopping in the hooks, but that can be done in future diffs. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/332 Test Plan: - Tested training, inference, validation runs, checked if events are happening in correct order with the new callback mechanism. Reviewed By: apsdehal Differential Revision: D22112333 Pulled By: vedanuj fbshipit-source-id: 07dbd42962691cf6b64bca2d53dc98f17150c853 30 June 2020, 03:58:20 UTC
back to top