Revision 0ee1127c81bf245225cd7db50d631e8677abaefc authored by Sasha Sheng on 08 February 2021, 10:50:33 UTC, committed by Facebook GitHub Bot on 08 February 2021, 10:52:20 UTC
Summary:
* pytorch lighting stub mostly involving training
* Tests for lightning trainer included
* built on top of the mmf grad accumulation fix: https://github.com/facebookresearch/mmf/pull/747
- [X] MVP 0. Training: Goal - Train a model from scratch and reach similar accuracy as using mmf_trainer
- [X] Setup the training pipeline: done
- [X] Training on the right device: done
- [X] Clip gradients: done
- [X] Optimizer: done
- [X] FP16 Support: done
- [X] LR scheduler (incl. warmup etc): done
- [X] testcase: train visual_bert on vqa from scratch fo 10 iterations, compare the value: done
- [x] tests included in this PR (tests are only done for pytorch lightning integration):
- [X] Vanilla Training w/o grad accumulate, make sure loss for 5 iters are the same between mmf and pl
- [X] Optimizer working as intended as a part of this PR.
- [X] `max_updates` and `max_epochs` calculation
- [x] Training with grad accumulate
- [x] Training with LR schedule achieves a different value compared to without LR schedule
- [x] Training with LR schedule for PL is the same as training with LR schedule for `mmf_tranier`
- [x] Training with gradient clipping make sure all grads are within the `grad_clipping` threshold
- [x] Training with gradient clipping is the same as training with gradient clipping for `mmf_trainer`
Pull Request resolved: https://github.com/facebookresearch/mmf/pull/748
Reviewed By: apsdehal, simran2905
Differential Revision: D26192869
Pulled By: ytsheng
fbshipit-source-id: 203a91e893d6b878bbed80ed84960dd059cfc90c
1 parent fc72ef0
README.md
<div align="center">
<img src="https://mmf.sh/img/logo.svg" width="50%"/>
</div>
#
<div align="center">
<a href="https://mmf.sh/docs">
<img alt="Documentation Status" src="https://readthedocs.org/projects/mmf/badge/?version=latest"/>
</a>
<a href="https://circleci.com/gh/facebookresearch/mmf">
<img alt="CircleCI" src="https://circleci.com/gh/facebookresearch/mmf.svg?style=svg"/>
</a>
</div>
---
MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-the-art vision and language models and has powered multiple research projects at Facebook AI Research. See full list of project inside or built on MMF [here](https://mmf.sh/docs/notes/projects).
MMF is powered by PyTorch, allows distributed training and is un-opinionated, scalable and fast. Use MMF to **_bootstrap_** for your next vision and language multimodal research project by following the [installation instructions](https://mmf.sh/docs/getting_started/installation). Take a look at list of MMF features [here](https://mmf.sh/docs/getting_started/features).
MMF also acts as **starter codebase** for challenges around vision and
language datasets (The Hateful Memes, TextVQA, TextCaps and VQA challenges). MMF was formerly known as Pythia. The next video shows an overview of how datasets and models work inside MMF. Checkout MMF's [video overview](https://mmf.sh/docs/getting_started/video_overview).
## Installation
Follow installation instructions in the [documentation](https://mmf.sh/docs/getting_started/installation).
## Documentation
Learn more about MMF [here](https://mmf.sh/docs).
## Citation
If you use MMF in your work or use any models published in MMF, please cite:
```bibtex
@misc{singh2020mmf,
author = {Singh, Amanpreet and Goswami, Vedanuj and Natarajan, Vivek and Jiang, Yu and Chen, Xinlei and Shah, Meet and
Rohrbach, Marcus and Batra, Dhruv and Parikh, Devi},
title = {MMF: A multimodal framework for vision and language research},
howpublished = {\url{https://github.com/facebookresearch/mmf}},
year = {2020}
}
```
## License
MMF is licensed under BSD license available in [LICENSE](LICENSE) file
Computing file changes ...
