https://github.com/facebookresearch/pythia
Revision 96ecb1128bf1786a8bba9038a7bdbbf407a1648c authored by Amanpreet Singh on 29 April 2021, 08:16:00 UTC, committed by Amanpreet Singh on 30 April 2021, 18:20:57 UTC
Summary: This PR adds support for audio and video modality encoders to MMF. These can be used in conjunction with MMFTransformer. An example config has been added to showcase the usage. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/879 Test Plan: Unit tests have been added. Reviewed By: ytsheng Differential Revision: D27804875 Pulled By: apsdehal fbshipit-source-id: 9f276dab2dc711fb8e5868a029f73c16083c1782
1 parent 231fb16
Tip revision: 96ecb1128bf1786a8bba9038a7bdbbf407a1648c authored by Amanpreet Singh on 29 April 2021, 08:16:00 UTC
[feat] Adds audio (resnet18) and video (r2plus1d18) encoders (#879)
[feat] Adds audio (resnet18) and video (r2plus1d18) encoders (#879)
Tip revision: 96ecb11
File | Mode | Size |
---|---|---|
interfaces | ||
transformers | ||
unit | ||
__init__.py | -rw-r--r-- | 1.0 KB |
alignment.py | -rw-r--r-- | 9.2 KB |
ban.py | -rw-r--r-- | 3.2 KB |
base_model.py | -rw-r--r-- | 12.8 KB |
butd.py | -rw-r--r-- | 7.1 KB |
cnn_lstm.py | -rw-r--r-- | 3.5 KB |
frcnn.py | -rw-r--r-- | 9.2 KB |
fusions.py | -rw-r--r-- | 7.2 KB |
lorra.py | -rw-r--r-- | 2.2 KB |
lxmert.py | -rw-r--r-- | 27.7 KB |
m4c.py | -rw-r--r-- | 21.9 KB |
m4c_captioner.py | -rw-r--r-- | 748 bytes |
mmbt.py | -rw-r--r-- | 25.0 KB |
mmf_bert.py | -rw-r--r-- | 16.0 KB |
mmf_transformer.py | -rw-r--r-- | 14.6 KB |
movie_mcan.py | -rw-r--r-- | 9.8 KB |
pythia.py | -rw-r--r-- | 18.6 KB |
top_down_bottom_up.py | -rw-r--r-- | 2.5 KB |
unimodal.py | -rw-r--r-- | 3.3 KB |
vilbert.py | -rw-r--r-- | 57.1 KB |
visdial_multi_modal.py | -rw-r--r-- | 3.3 KB |
visual_bert.py | -rw-r--r-- | 23.5 KB |
Computing file changes ...