https://github.com/facebookresearch/pythia
- HEAD
- refs/heads/0.1
- refs/heads/additional_validate
- refs/heads/airstore
- refs/heads/audio_video_encoders
- refs/heads/config_mmft_doc
- refs/heads/debug_ln_masked_lm
- refs/heads/dependabot/npm_and_yarn/website/ansi-regex-4.1.1
- refs/heads/dependabot/npm_and_yarn/website/http-cache-semantics-4.1.1
- refs/heads/dependabot/npm_and_yarn/website/terser-5.16.2
- refs/heads/dependabot/npm_and_yarn/website/webpack-5.76.1
- refs/heads/es
- refs/heads/fix
- refs/heads/fix_base_model
- refs/heads/fix_ci
- refs/heads/fix_circleci_tests
- refs/heads/fix_lint
- refs/heads/fixblack
- refs/heads/fsdp
- refs/heads/fsdp_asg
- refs/heads/fsdp_fairseq
- refs/heads/fsdp_support
- refs/heads/gh-pages
- refs/heads/gh/ebsmothers/1/base
- refs/heads/gh/ryan-qiyu-jiang/30/base
- refs/heads/gh/ryan-qiyu-jiang/30/head
- refs/heads/gh/ryan-qiyu-jiang/30/orig
- refs/heads/gh/ryan-qiyu-jiang/32/base
- refs/heads/gh/ryan-qiyu-jiang/32/head
- refs/heads/gh/ryan-qiyu-jiang/32/orig
- refs/heads/gh/ryan-qiyu-jiang/39/base
- refs/heads/gh/ryan-qiyu-jiang/39/head
- refs/heads/gh/ryan-qiyu-jiang/39/orig
- refs/heads/gh/ryan-qiyu-jiang/40/base
- refs/heads/gh/ryan-qiyu-jiang/40/head
- refs/heads/gh/ryan-qiyu-jiang/40/orig
- refs/heads/gh/ryan-qiyu-jiang/42/base
- refs/heads/gh/ryan-qiyu-jiang/42/head
- refs/heads/gh/ryan-qiyu-jiang/42/orig
- refs/heads/gh/ytsheng/2/base
- refs/heads/gh/ytsheng/2/orig
- refs/heads/gh/ytsheng/21/base
- refs/heads/gh/ytsheng/21/orig
- refs/heads/gh/ytsheng/3/base
- refs/heads/gh/ytsheng/3/orig
- refs/heads/gh/ytsheng/4/base
- refs/heads/gh/ytsheng/5/base
- refs/heads/gh/ytsheng/6/base
- refs/heads/gh/ytsheng/7/base
- refs/heads/gh/ytsheng/8/base
- refs/heads/hydra
- refs/heads/itm
- refs/heads/main
- refs/heads/mask_key_mmft
- refs/heads/mm_alignamet
- refs/heads/mmf_interactive
- refs/heads/mmft_output_key
- refs/heads/multitask_training
- refs/heads/notebooks
- refs/heads/pl_upgrade
- refs/heads/project/cycle-consistency
- refs/heads/project/m4c
- refs/heads/pt_19
- refs/heads/skip_optimizer_update
- refs/heads/stable_mmft
- refs/heads/v0.4
- refs/heads/vilbert_multimodal
- refs/heads/vilbert_multitask
- refs/heads/vilbert_multitask_flickr30k
- refs/heads/vilbert_multitask_visual7w
- refs/heads/xla-checkpoint-fix
- refs/tags/v0.3
- refs/tags/v0.3.1
Take a new snapshot of a software origin
If the archived software origin currently browsed is not synchronized with its upstream version (for instance when new commits have been issued), you can explicitly request Software Heritage to take a new snapshot of it.
Use the form below to proceed. Once a request has been submitted and accepted, it will be processed as soon as possible. You can then check its processing state by visiting this dedicated page.
Processing "take a new snapshot" request ...
Permalinks
To reference or cite the objects present in the Software Heritage archive, permalinks based on SoftWare Hash IDentifiers (SWHIDs) must be used.
Select below a type of object currently browsed in order to display its associated SWHID and permalink.
| Revision | Author | Date | Message | Commit Date |
|---|---|---|---|---|
| 829ea29 | Amanpreet Singh | 17 June 2021, 20:02:27 UTC | [feat] Support custom mask key in MMF Transformer Now, you can specify custom mask key for a modality using `mask_key` attribute. This also fixes an issue which caused MMFT to fail when text modality was not present. | 17 June 2021, 20:04:09 UTC |
| b08ac7a | KMarino | 17 June 2021, 02:19:08 UTC | [feat] Adding KRISP, additions to OKVQA dataset (#750) Summary: Adding KRISP Paper to mmf repo Additional changes to use image features for OKVQA Pull Request resolved: https://github.com/facebookresearch/mmf/pull/750 Reviewed By: ytsheng, vedanuj Differential Revision: D29082706 Pulled By: apsdehal fbshipit-source-id: 5eb3346291f6de8eaf44385d7a6cba52d317b8d4 | 17 June 2021, 02:20:03 UTC |
| 7705b7f | Licheng Yu | 15 June 2021, 02:58:13 UTC | [mmf_ads][sweep] sweep pretrain+finetune large model (#183) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/183 Sweep finetuning from a pre-trained large model Differential Revision: D29056437 fbshipit-source-id: 22214d8d3c6958ce4a4a8a0fcef0455efcf0be34 | 15 June 2021, 02:59:07 UTC |
| 9d5222a | Mengjiao MJ Wang | 15 June 2021, 00:34:13 UTC | [feat] add resnet18 as encoder (#970) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/970 Adding resnet-18 image encoder Reviewed By: apsdehal Differential Revision: D28267636 fbshipit-source-id: 49ad3860b7abfa7d0697be13a1369500337e9ed1 | 15 June 2021, 00:35:02 UTC |
| 322bd33 | Sasha Sheng | 12 June 2021, 16:58:11 UTC | [feat] PL last validation after training (#933) Summary: * this implements the last validation functionality after training since it was added in PL master recently. * updated the tests to check for the last validation result. * still waiting on PL to fix its tariner.global_step inconsistency * two reports are being kept for now until we figure out a better way to consolidate them. They are being kept in the lightning module for calculating the early stopping and saving the checkpoint (PR coming soon); they are being kept in the loop callback for logging purposes. Eventually, we may move all of them into the lightning module. But I am submitting the PRs one step at the time for now. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/933 Reviewed By: apsdehal, simran2905 Differential Revision: D28329041 Pulled By: ytsheng fbshipit-source-id: 60f014fa37556a6d15645d5cfe8b1442c12b9aa2 | 12 June 2021, 16:58:59 UTC |
| 7cc07dc | dependabot[bot] | 12 June 2021, 00:16:20 UTC | Bump url-parse from 1.4.7 to 1.5.1 in /website (#932) Summary: Bumps [url-parse](https://github.com/unshiftio/url-parse) from 1.4.7 to 1.5.1. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/unshiftio/url-parse/commit/eb6d9f51e395b7e47bf2594e457d541db21c713b"><code>eb6d9f5</code></a> [dist] 1.5.1</li> <li><a href="https://github.com/unshiftio/url-parse/commit/750d8e8a9d45dbce9ff09759f0fe4564cdd47d74"><code>750d8e8</code></a> [fix] Fixes relative path resolving <a href="https://github-redirect.dependabot.com/unshiftio/url-parse/issues/199">https://github.com/facebookresearch/mmf/issues/199</a> <a href="https://github-redirect.dependabot.com/unshiftio/url-parse/issues/200">https://github.com/facebookresearch/mmf/issues/200</a> (<a href="https://github-redirect.dependabot.com/unshiftio/url-parse/issues/201">https://github.com/facebookresearch/mmf/issues/201</a>)</li> <li><a href="https://github.com/unshiftio/url-parse/commit/3ac777474ba5dc48a7e33771cbb311fc6f69bef8"><code>3ac7774</code></a> [test] Make test consistent for browser testing</li> <li><a href="https://github.com/unshiftio/url-parse/commit/267a0c6f7ef1a58271be61611c5103daace602c9"><code>267a0c6</code></a> [dist] 1.5.0</li> <li><a href="https://github.com/unshiftio/url-parse/commit/d1e7e8822f26e8a49794b757123b51386325b2b0"><code>d1e7e88</code></a> [security] More backslash fixes (<a href="https://github-redirect.dependabot.com/unshiftio/url-parse/issues/197">https://github.com/facebookresearch/mmf/issues/197</a>)</li> <li><a href="https://github.com/unshiftio/url-parse/commit/d99bf4cf259b7378c855f786edc253e70405ffdc"><code>d99bf4c</code></a> [ignore] Remove npm-debug.log from .gitignore</li> <li><a href="https://github.com/unshiftio/url-parse/commit/422c8b5e4cac6a79cd35b4e86731476dcbeec7e4"><code>422c8b5</code></a> [pkg] Replace nyc with c8</li> <li><a href="https://github.com/unshiftio/url-parse/commit/933809d630c7b21399b4e5df59fccccd80033b21"><code>933809d</code></a> [pkg] Move coveralls to dev dependencies</li> <li><a href="https://github.com/unshiftio/url-parse/commit/190b2168035899a2a88f2dc2625963bf7e2f338f"><code>190b216</code></a> [pkg] Add .npmrc</li> <li><a href="https://github.com/unshiftio/url-parse/commit/ce3783f4ea25753cfa36376769c14e4e2fe6ea80"><code>ce3783f</code></a> [test] Do not test on all available versions of Edge and Safari</li> <li>Additional commits viewable in <a href="https://github.com/unshiftio/url-parse/compare/1.4.7...1.5.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/932 Reviewed By: apsdehal Differential Revision: D28430674 Pulled By: vedanuj fbshipit-source-id: 7691d843940db4685202618bda30cc99a55e00d3 | 12 June 2021, 00:17:09 UTC |
| 08b2238 | dependabot[bot] | 12 June 2021, 00:11:39 UTC | Bump lodash from 4.17.19 to 4.17.21 in /website (#936) Summary: Bumps [lodash](https://github.com/lodash/lodash) from 4.17.19 to 4.17.21. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/lodash/lodash/commit/f299b52f39486275a9e6483b60a410e06520c538"><code>f299b52</code></a> Bump to v4.17.21</li> <li><a href="https://github.com/lodash/lodash/commit/c4847ebe7d14540bb28a8b932a9ce1b9ecbfee1a"><code>c4847eb</code></a> Improve performance of <code>toNumber</code>, <code>trim</code> and <code>trimEnd</code> on large input strings</li> <li><a href="https://github.com/lodash/lodash/commit/3469357cff396a26c363f8c1b5a91dde28ba4b1c"><code>3469357</code></a> Prevent command injection through <code>_.template</code>'s <code>variable</code> option</li> <li><a href="https://github.com/lodash/lodash/commit/ded9bc66583ed0b4e3b7dc906206d40757b4a90a"><code>ded9bc6</code></a> Bump to v4.17.20.</li> <li><a href="https://github.com/lodash/lodash/commit/63150ef7645ac07961b63a86490f419f356429aa"><code>63150ef</code></a> Documentation fixes.</li> <li><a href="https://github.com/lodash/lodash/commit/00f0f62a979d2f5fa0287c06eae70cf9a62d8794"><code>00f0f62</code></a> test.js: Remove trailing comma.</li> <li><a href="https://github.com/lodash/lodash/commit/846e434c7a5b5692c55ebf5715ed677b70a32389"><code>846e434</code></a> Temporarily use a custom fork of <code>lodash-cli</code>.</li> <li><a href="https://github.com/lodash/lodash/commit/5d046f39cbd27f573914768e3b36eeefcc4f1229"><code>5d046f3</code></a> Re-enable Travis tests on <code>4.17</code> branch.</li> <li><a href="https://github.com/lodash/lodash/commit/aa816b36d402a1ad9385142ce7188f17dae514fd"><code>aa816b3</code></a> Remove <code>/npm-package</code>.</li> <li>See full diff in <a href="https://github.com/lodash/lodash/compare/4.17.19...4.17.21">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~bnjmnt4n">bnjmnt4n</a>, a new releaser for lodash since your current version.</p> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/936 Reviewed By: apsdehal Differential Revision: D28430679 Pulled By: vedanuj fbshipit-source-id: 017157083dc80a6d705fbe1c2964847cf4b43396 | 12 June 2021, 00:13:05 UTC |
| f8e424a | dependabot[bot] | 12 June 2021, 00:09:39 UTC | Bump ws from 6.2.1 to 6.2.2 in /website (#957) Summary: Bumps [ws](https://github.com/websockets/ws) from 6.2.1 to 6.2.2. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/websockets/ws/commits">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/957 Reviewed By: apsdehal Differential Revision: D28995838 Pulled By: vedanuj fbshipit-source-id: c5acd84ec6c66f4c737774f7c29ab9403174d2e3 | 12 June 2021, 00:10:30 UTC |
| 778893c | dependabot[bot] | 12 June 2021, 00:06:17 UTC | Bump merge-deep from 3.0.2 to 3.0.3 in /website (#963) Summary: Bumps [merge-deep](https://github.com/jonschlinkert/merge-deep) from 3.0.2 to 3.0.3. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/jonschlinkert/merge-deep/commit/628ff47c9d824ccf21adf9a2b7cc6b74632e11a1"><code>628ff47</code></a> 3.0.3</li> <li><a href="https://github.com/jonschlinkert/merge-deep/commit/cfbe20ccdb00255b711de57e37ed8ce9f109ef3f"><code>cfbe20c</code></a> run verb to generate README documentation</li> <li><a href="https://github.com/jonschlinkert/merge-deep/commit/e370968581413a2e5ffdbbf7c2f5094e0e0b3861"><code>e370968</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/jonschlinkert/merge-deep/issues/17">https://github.com/facebookresearch/mmf/issues/17</a> from jonschlinkert/key-properties</li> <li><a href="https://github.com/jonschlinkert/merge-deep/commit/393e2cbaeacf54e77a307c3620a00f0ac057b8d5"><code>393e2cb</code></a> adding a test to ensure using merge-deep for inheritance still works</li> <li><a href="https://github.com/jonschlinkert/merge-deep/commit/c39b16134a6a9704be2e661b49b92e8561f10d90"><code>c39b161</code></a> add test to ensure constructor is not cloned</li> <li><a href="https://github.com/jonschlinkert/merge-deep/commit/11e5dd56de8a6aed0b1ed022089dbce6968d82a5"><code>11e5dd5</code></a> add isValidKey function to ensure only valid keys are merged</li> <li>See full diff in <a href="https://github.com/jonschlinkert/merge-deep/compare/3.0.2...3.0.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/963 Reviewed By: apsdehal Differential Revision: D28995854 Pulled By: vedanuj fbshipit-source-id: 0e143e6746df8e5120a2a74633a0b6ea0d781018 | 12 June 2021, 00:09:06 UTC |
| 7dfad57 | dependabot[bot] | 12 June 2021, 00:05:25 UTC | Bump hosted-git-info from 2.8.8 to 2.8.9 in /website (#937) Summary: Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md">hosted-git-info's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9">2.8.9</a> (2021-04-07)</h2> <h3>Bug Fixes</h3> <ul> <li>backport regex fix from <a href="https://github-redirect.dependabot.com/npm/hosted-git-info/issues/76">https://github.com/facebookresearch/mmf/issues/76</a> (<a href="https://github.com/npm/hosted-git-info/commit/29adfe5">29adfe5</a>), closes <a href="https://github-redirect.dependabot.com/npm/hosted-git-info/issues/84">https://github.com/facebookresearch/mmf/issues/84</a></li> </ul> <p><!-- raw HTML omitted --><!-- raw HTML omitted --></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/npm/hosted-git-info/commit/8d4b3697d79bcd89cdb36d1db165e3696c783a01"><code>8d4b369</code></a> chore(release): 2.8.9</li> <li><a href="https://github.com/npm/hosted-git-info/commit/29adfe5ef789784c861b2cdeb15051ec2ba651a7"><code>29adfe5</code></a> fix: backport regex fix from <a href="https://github-redirect.dependabot.com/npm/hosted-git-info/issues/76">https://github.com/facebookresearch/mmf/issues/76</a></li> <li>See full diff in <a href="https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~nlf">nlf</a>, a new releaser for hosted-git-info since your current version.</p> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/937 Reviewed By: apsdehal Differential Revision: D28430663 Pulled By: vedanuj fbshipit-source-id: 167c5d80e1148d1c244f01940aa85adf81c6ef38 | 12 June 2021, 00:06:43 UTC |
| dce69a1 | Vedanuj Goswami | 11 June 2021, 23:39:25 UTC | [feat] AIRStore Dataset Integration (#918) Summary: - Add Airstore dataset - Add config for running masked coco training with Airstore Pull Request resolved: https://github.com/facebookresearch/mmf/pull/918 Test Plan: - Tested with Masked COCO training Reviewed By: apsdehal Differential Revision: D29050028 Pulled By: vedanuj fbshipit-source-id: 1e5b320778418a832507a5fbe6aa16d214f36c12 | 11 June 2021, 23:40:22 UTC |
| b00223d | Ronghang Hu | 10 June 2021, 09:44:05 UTC | [fix] set logging level according to training.logger_level (#959) Summary: Correctly set logging level in the logger according to training.logger_level. Also fix lint errors in other files to pass the tests. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/959 Test Plan: tested locally. Reviewed By: vedanuj Differential Revision: D28925702 Pulled By: ronghanghu fbshipit-source-id: e42e8eb559b2b32f4e8c7dda9fc2a3a424f63eda | 10 June 2021, 09:44:44 UTC |
| 6777ca4 | Jie Lei 雷杰 | 08 June 2021, 17:43:36 UTC | fix fp16 overflow issue (#962) Summary: "-1e9" induces "RuntimeError: value cannot be converted to type at::Half without overflow:" when fp16=true, replace it with "-10000.0" Tested locally Thanks for your contribution! If you're sending a large PR (e.g., >50 lines), please open an issue first about the feature/bug, and indicate how you want to contribute. Use [contributing guidelines](https://github.com/facebookresearch/mmf/tree/master/.github/CONTRIBUTING.md) before opening up the PR to follow MMF style guidelines. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/962 Reviewed By: vedanuj Differential Revision: D28963671 Pulled By: jayleicn fbshipit-source-id: f8dcdd28eeccc9d2b19af4cd17710c869feb0eea | 08 June 2021, 17:44:23 UTC |
| 5265e1c | Amanpreet Singh | 08 June 2021, 06:31:18 UTC | [feat,fix] Random init bert encocder, other QoL fixes (#941) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/941 - Add dataset_type property to base_dataset class to avoid some failures in dataset builders - broadcast fp16 in OSS - Fix NCCL error happening in sweep script - is_correct if not present in processor output defaults to True This changes are required for running fairseq_mlm. Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/178 Reviewed By: vedanuj Differential Revision: D28393021 Pulled By: apsdehal fbshipit-source-id: c31bcff57d1ce8bde71a38e7e99cc70ecb5a7c13 | 08 June 2021, 06:32:06 UTC |
| 1bb7b75 | Sasha Sheng | 08 June 2021, 06:23:58 UTC | [feat][6/N] clip accuracy (#165) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/165 Test Plan: Run `pytest tests` to see the metrics test cases pass Reviewed By: apsdehal Differential Revision: D27457530 Pulled By: ytsheng fbshipit-source-id: ed82a88f3fc21a881c9d4d364a9efaba624cf95e | 08 June 2021, 06:25:29 UTC |
| 4694a4e | Madian Khabsa | 04 June 2021, 19:20:48 UTC | [MMF][fix] fix python version comparison (#958) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/958 Move from string based comparison to major/minor integer based comparison. Searched for `torch.__version__` in repo to find all uses Reviewed By: vedanuj Differential Revision: D28900525 fbshipit-source-id: b6c069ffdba9b849c58bf6f50f5d453e0edb68ee | 04 June 2021, 19:22:01 UTC |
| 10cc2b7 | dependabot[bot] | 03 June 2021, 18:21:10 UTC | Bump dns-packet from 1.3.1 to 1.3.4 in /website (#951) Summary: Bumps [dns-packet](https://github.com/mafintosh/dns-packet) from 1.3.1 to 1.3.4. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/mafintosh/dns-packet/commit/ebdf849da5dc0d96836e87628349776c623c5be7"><code>ebdf849</code></a> 1.3.4</li> <li><a href="https://github.com/mafintosh/dns-packet/commit/ac578722f2707310b841b65aae61d6332f8882a1"><code>ac57872</code></a> move all allocUnsafes to allocs for easier maintenance</li> <li><a href="https://github.com/mafintosh/dns-packet/commit/c64c9507e51532c9e9a3cbefa146a134ecc025fd"><code>c64c950</code></a> 1.3.3</li> <li><a href="https://github.com/mafintosh/dns-packet/commit/0598ba19d18da4568b32415e60a9629061b3c45c"><code>0598ba1</code></a> fix .. in encodingLength</li> <li><a href="https://github.com/mafintosh/dns-packet/commit/010aedb33c1ee8c3f558db5249c1d46e2bd7a101"><code>010aedb</code></a> 1.3.2</li> <li><a href="https://github.com/mafintosh/dns-packet/commit/0d0d593f8df4e2712c43957a6c62e95047f12b2d"><code>0d0d593</code></a> backport encodingLength fix to v1</li> <li>See full diff in <a href="https://github.com/mafintosh/dns-packet/compare/v1.3.1...v1.3.4">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/951 Reviewed By: ytsheng Differential Revision: D28839295 Pulled By: vedanuj fbshipit-source-id: 77ef7f8af869fa255bf410a272112cf9fe893e75 | 03 June 2021, 18:22:09 UTC |
| 5d94327 | Jun Chen | 03 June 2021, 06:45:35 UTC | [feat] Implement knowledge distillation in MMF (#956) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/956 Add a general knowledge distillation in MMF # High-level Design * Knowledge distillation model contains teacher and student models. Teacher model is used to generate soft targets and let student model mimic. * Add a processor which can process text and feature of teacher and student models. The text/feature processors can be different. * In yaml config, student model still uses the name like `text_processor` in dataset_config. `teacher_text_processor` or `teacher_feature_key` can be added to specify teacher's processor or feature_key. Teacher model is built and loaded from `modal_config.kd.teacher.pretrained_checkpoint` and its parameters can be frozen or tunable. Student config has the same format as teacher's and supports loading pretrained model. * Support three distillation losses: MSELoss, CosineEmbeddingLoss, and KLDivergence. The final loss is a weighted sum of distillation and metric losses. # Summary of experiment results 1. LR set as 1e-4 to 1e-3 2. KL divergence as distillation loss is the best 3. Classification loss weight should be small 4. Different teacher architecture doesn't really matter 5. It doesn't matter whether student parameters are pretrained or not 6. If image features are the same in teacher and student, extra data is not needed to get similar performance 7. Otherwise, unlabeled data will help student model performance Reviewed By: vedanuj Differential Revision: D27726297 fbshipit-source-id: 37673782265aa1e80af0172b974ae430d6fd7a6e | 03 June 2021, 06:46:37 UTC |
| c2af5c1 | Rui Hou | 02 June 2021, 05:16:15 UTC | [feat] allow modules to have mismatch dimensions when loading pretrained checkpoints (#955) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/955 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/182 Make it possible to skip dimension mismatch weight copying during pretrained checkpoint loading if users specify so. One use case is to add new modalities (xray video embedding) into the pretrained checkpoint so that things like token_type embedding can be changed. Reviewed By: apsdehal Differential Revision: D28703466 fbshipit-source-id: 95a1abc115b3095f658d2fe9c89440f571568b36 | 02 June 2021, 05:17:17 UTC |
| b37493a | Rui Hou | 02 June 2021, 04:55:52 UTC | [feat] allow different lr to be set for encoders (image encoder) (#181) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/181 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/953 This diff allows to set different lr to different encoders such as image encoder. Reviewed By: apsdehal Differential Revision: D28700114 fbshipit-source-id: c5e44cde08c95ebde803ce80ac166b32f7ac7249 | 02 June 2021, 04:57:04 UTC |
| 25d69c8 | Christopher Dewan | 21 May 2021, 22:44:17 UTC | [PostRay] Create a Recurring WPIE Eval that skips the sweep launcher 1/N (#179) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/179 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/950 Created a new flow workflow that performs the same operations as the current launcher, but does it as part of flow execution so it can be launched with flow-cli (or indeed, as a recurring run) The prefix field defaults to having a date/time macro in it which should allow us to avoid collisions in the recurring run Reviewed By: madian9, apsdehal Differential Revision: D28608702 fbshipit-source-id: 9b43f3aa6ea34893bf9d172cf37e2e4e8ba1d25f | 21 May 2021, 22:44:54 UTC |
| da8287a | Jun Chen | 21 May 2021, 00:03:09 UTC | [MMC] Experiment on attribute new taxonomy Summary: Experiments on commerce attribute with new taxonomy Modifications 1. Fix soft label masking error when label contains `ignore_index` 2. Compute AP even if some labels are missing during validation 3. Add metric `accuracy_with_unknown` for missing category Reviewed By: lichengunc Differential Revision: D28346114 fbshipit-source-id: a8f76eb29a3fdb9f165aecbd66428511ec154a50 | 21 May 2021, 00:04:06 UTC |
| 3183b13 | Ronghang Hu | 20 May 2021, 17:47:05 UTC | [feat] add project docs to UniT model (#920) Summary: Add a project page to UniT in the MMF website. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/920 Test Plan: tested website building internally Reviewed By: vedanuj Differential Revision: D28137781 Pulled By: ronghanghu fbshipit-source-id: 9126b013116633921f439699ec8e6a4a9359d279 | 20 May 2021, 17:47:51 UTC |
| 7bca9c1 | Simran Motwani | 17 May 2021, 21:09:24 UTC | [mmf] Add usage logging to components (#938) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/938 Adding usage logging to MMF components for analytics Reviewed By: ytsheng Differential Revision: D28309104 fbshipit-source-id: c9dfd619ceba6d9d9c87df4ab86cf1773f51bde6 | 17 May 2021, 21:10:24 UTC |
| 44ae471 | Amanpreet Singh | 12 May 2021, 22:33:17 UTC | [fix] Memory leak in test reporter; detach report in inference (#939) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/939 In recent PR, we started adding to report in all processors to better support TPUs. Unfortunately, the data collected in test reporter was not continuously flushed in cases other than master. This PR fixes. Other than that, we now detach the report as well in prediction loop. Reviewed By: rayhou0710 Differential Revision: D28338749 fbshipit-source-id: 850ce1b5b77d992a9357a60cbf107b42ff6e81d5 | 12 May 2021, 22:34:25 UTC |
| 957087b | Sasha Sheng | 12 May 2021, 08:32:17 UTC | [fix,feat] fix lightning requirements + add leak hook (#940) Summary: * Lightning got updated and they also updated the internal tests, but they did not update the OSS dependency. We should communicate it to them so that they can also the OSS side. Or if there is an automatic way of doing this, it would be even better. * This diff adds a pytest hook to check for memory leaks being caused. This just adds printing statements. {F615127206} Pull Request resolved: https://github.com/facebookresearch/mmf/pull/940 Reviewed By: apsdehal Differential Revision: D28375742 Pulled By: ytsheng fbshipit-source-id: d6d6978ffaec3a70730a1126761b8e9e1190bae3 | 12 May 2021, 08:32:57 UTC |
| 51bc339 | Luis Perez | 12 May 2021, 00:37:44 UTC | Synchronize PyTorchLightning/pytorch-lightning (revision 7b283e3c@master) to github/third-party/PyTorchLightning/pytorch-lightning Summary: # Manual - remove fixme's in `model_checkpoint.py`, `parameter_monitor.py`, `test_quantization.py`, and `speed_monitor.py` now that `Trainer` is properly annotated. - update `test_quantization.py` to `trainer.train_loop.global_step` instead of `trainer.global_step` which is a read-only. - update `loop_callback.py` to read from `train_loop` for `batch_idx` (which is no longer available). # Automatic ### New commit log messages 7b283e3c Bugfix/Multiple dataloaders (#7433) d7c44cc6 Docs: sync chlog 1.3.1 (#7478) fdf50a5e Mark certain Trainer APIs as protected (#7420) ad9118f0 remove trainer hidden state | sanity refactor [1 / n] (#7437) 4a1134db Log epoch metrics before firing the `on_evaluation_end` hook (#7272) b65ae794 Automatically check `DataModule.has_{setup,teardown,prepare_data}` [2/2] (#7238) 8660d8cf [pre-commit.ci] pre-commit autoupdate (#7475) f6fe715e Fix Sphinx argument deprecation (#7464) Reviewed By: shuyingsunshine21 Differential Revision: D28353491 fbshipit-source-id: 98b87d99e2f09b47b07270858fcbdb5d5299730b | 12 May 2021, 00:38:28 UTC |
| 8447557 | Sasha Sheng | 10 May 2021, 23:05:16 UTC | [fix] isort broken on master (#934) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/934 Reviewed By: apsdehal Differential Revision: D28317799 Pulled By: ytsheng fbshipit-source-id: 0e69b7dec6c0e476ea9ab71326b8b32dd1fe4c4e | 10 May 2021, 23:06:10 UTC |
| 38fab0b | Rui Hou | 09 May 2021, 00:44:49 UTC | [feat] make it optional to tie mlm head weights to text_encoder embedding weights (#172) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/172 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/874 Make it possible to have the mlm head weights tied to text_encoder embedding weights in case we have a separate text encoder for encoding text modalities. Reviewed By: apsdehal Differential Revision: D27756957 fbshipit-source-id: ce26e79ca9930127f36c3c2adfe280e3c67ac23a | 09 May 2021, 00:45:41 UTC |
| dbe8dec | Sasha Sheng | 08 May 2021, 21:26:21 UTC | [refactor][4/N]: allow download to work for manifold files (#173) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/173 * adds a feature in download.py to allow it to work for manifold * onboards clip_processor to work with all three config below: * local on disk file * manifold file * http file Reviewed By: vedanuj Differential Revision: D27760358 fbshipit-source-id: 1b7b8eff09a21e8afc48971d69d46df18a8ced6b | 08 May 2021, 21:27:00 UTC |
| 2147104 | Rui Hou | 08 May 2021, 04:05:19 UTC | [postray] move around modality keys when searching for image channel Summary: In the use case of WPIE in D28278151, we take multiple image type inputs such as raw image, xray, xray_oc, cluster_fit, etc. Move modality to the first place of _check_keys_for_modality() so that we first search for the modality name. Otherwise, all image type encoders will pick up the "image" key. Reviewed By: apsdehal Differential Revision: D28306631 fbshipit-source-id: 021cb6300e228431797996321d28bbd8ebd2865d | 08 May 2021, 04:06:04 UTC |
| e2d2742 | Sasha Sheng | 07 May 2021, 06:03:00 UTC | [fix] fall back to None for max_updates and num_updates on trainer (#924) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/924 Reviewed By: apsdehal Differential Revision: D28179972 Pulled By: ytsheng fbshipit-source-id: 37cc47b6a81d375558eec522bd8724fad6324513 | 07 May 2021, 06:03:45 UTC |
| 26cd2ea | Sasha Sheng | 06 May 2021, 03:05:35 UTC | [fix] sound file import (#929) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/929 fix: https://fb.workplace.com/groups/574291956570984/permalink/732930907373754/ Reviewed By: apsdehal Differential Revision: D28242842 fbshipit-source-id: cd67eb71ab2739396d0bbc8b47f7b8eebaca0ab3 | 06 May 2021, 03:06:18 UTC |
| d2a2e91 | Amanpreet Singh | 06 May 2021, 02:25:23 UTC | [fix] Assign config to instance in setup as well (#928) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/928 Followup to https://github.com/facebookresearch/mmf/pull/921 for datasets which don't inherit from MMFDatasetBuilder Reviewed By: ytsheng Differential Revision: D28238562 fbshipit-source-id: 2502800fbe72a81b578aacbb38faebb9364c813f | 06 May 2021, 02:28:18 UTC |
| 90c1e74 | Rui Hou | 06 May 2021, 00:21:42 UTC | [fix] get_optimizer_parameters in BaseTransformer (#927) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/927 Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/177 Fix the functionality of assigning a different lr to task heads. Reviewed By: vedanuj Differential Revision: D28112762 fbshipit-source-id: f7038594ff7451031e557bc81e4bf3adbff84da0 | 06 May 2021, 00:22:35 UTC |
| 18534f8 | Amanpreet Singh | 05 May 2021, 09:37:28 UTC | [feat] Charades dataset along with video and audio processors (#894) Summary: This PR adds support for Charades dataset along with general purpose video and audio processors Pull Request resolved: https://github.com/facebookresearch/mmf/pull/894 Test Plan: ``` python mmf_cli/run.py config=projects/mmf_transformer/configs/charades/direct.yaml run_type=train_val dataset=charades model=mmf_transformer training.batch_size=4 training.num_workers=1 training.find_unused_parameters=True training.log_interval=100 ``` Reviewed By: ytsheng Differential Revision: D28124494 Pulled By: apsdehal fbshipit-source-id: b2c11598872baef5cc279d653176d2b45d0e45eb | 05 May 2021, 09:38:22 UTC |
| 0505b85 | Amanpreet Singh | 05 May 2021, 04:56:51 UTC | [fix] XLA issues, test reporter barrier, oc and torch warnings (#922) Summary: - Return false for is_xla in case it is not registered for correct drop_last setting in dataloader - Always call format_for_prediction for all workers to avoid hangs in TPU and NCCLs - Move .pretty use in OC to OmegaConf.to_yaml - Remove `check_reduction` parameter from DDP init Pull Request resolved: https://github.com/facebookresearch/mmf/pull/922 Test Plan: Tested locally with changes on charades branch Reviewed By: vedanuj Differential Revision: D28156890 Pulled By: apsdehal fbshipit-source-id: 8d4156f050dd09e4201673dcc030f242d8bd4df7 | 05 May 2021, 04:57:43 UTC |
| fe714b3 | Licheng Yu | 04 May 2021, 07:01:16 UTC | [mmf][PR][fix] Fix roberta_processor and transformer_processor (#923) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/923 **BERT encodes:** - **single sequence: ``[CLS] X [SEP]``** - **pair of sequences: ``[CLS] A [SEP] B [SEP]``** **RoBERTa and SPM encode:** - **single sequence: ``<s> X </s>``** - **pair of sequences: ``<s> A </s> </s> B </s>``** The original implementation would cause: 1) if a sentence is longer than max_seq_len, it will be chunked without eos token, i.e., `<s> X` instead of `<s> X </s>`. 2) the masking strategy is not correctly supported: 2a) no correct `<mask>` token; 2b) bos and eos token could be masked out in the function of `random_word()`. In this diff, we made the following changes: 1) Add `<mask>` token and its mapping to `SentencePieceTokenizer`. 2) Add `MaskedRobertaTokenizer` class and our `SentencePieceTokenizer` would be its subclass. 3) Make sure `_truncate_seq_pair` will return sentence of `<s> x </s>` or a pair of sentence of `<s> x </s> </s> y </s>`. Previously the last `</s>` would be popped if the input was a too long sentence. 4) Add tests of too long sentence, two sentence, and masked sentence. For making `SentencePieceTokenizer` a subclass of `MaskedRobertaTokenizer`, we simply did a small surgery of the returned SPM's tokenization result, which removes the pre-padded `<s>` and `</s>` from the input sentence. We will add them back in `_convert_to_indices`. Reviewed By: apsdehal Differential Revision: D28025537 fbshipit-source-id: 24b4dbe1118734f7bb5942d2e3deba190fac9698 | 04 May 2021, 07:02:00 UTC |
| 8a01258 | Amanpreet Singh | 03 May 2021, 22:30:09 UTC | [fix] multinode jobs after recent lightning update (#921) Summary: After Sasha's update of pytorch lightning on MMF master, it broke MMF codebase for multinode job. The root problem to PR https://github.com/PyTorchLightning/pytorch-lightning/pull/6802. The assumption that SLURM_PROCID points to worker rank is wrong as some frameworks launch their own processes later using multiprocessing spawn and have ntasks_per_node=1 set. This means that first node will have procid = 0, second node will have procid = 1 set and so on. Now, since this is used in prepare_data masking in LightningDataModule, this leads to it running on all workers on first node and thus causing inconsistencies. Now, this leads to prepare_data being called on all workers on first node instead of rank zero. Specifically, the barrier call in prepare_data, is called on first node workers but not on others leading to block later on. This PR fixes this by ensuring on our side that we only call prepare_data on rank zero. Furthermore, this can cause further confusion, we remove sync barrier calls from download as well. Users are now supposed to handle is_master checks on their own. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/921 Test Plan: Tested in multinode settings. Reviewed By: vedanuj Differential Revision: D28156855 Pulled By: apsdehal fbshipit-source-id: 4e0dd5317e15153f558d34c6951a89299602454f | 03 May 2021, 22:31:47 UTC |
| 1cc6b08 | Ronghang Hu | 01 May 2021, 03:18:04 UTC | [feat] allow attribute prediction in UniT; fix UniT zoo ckpt yaml config (#910) Summary: - allow predicting attributes in Visual Genome detection -- run with `model_config.unit.predict_attributes=True` to write attributes into json prediction files - also fix the config.yaml files for the UniT checkpoints - visualizing predicted boxes and attributes: https://gist.github.com/ronghanghu/47053b7e58460d48986345fb9027b460#file-visualize_detection_vg_outputs_with_attr-ipynb Pull Request resolved: https://github.com/facebookresearch/mmf/pull/910 Test Plan: tested locally and verified attribute outputs Reviewed By: apsdehal Differential Revision: D28101483 Pulled By: ronghanghu fbshipit-source-id: f4db66c562d2867c473347e13db042cc9b0fd462 | 01 May 2021, 03:18:58 UTC |
| d30dad3 | Aapo Kyrola | 30 April 2021, 20:26:35 UTC | [mmf/debuggability] longer timeout on first batch (#919) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/919 Timeout on first batch should be much longer, because workers might start at different times. Reviewed By: madian9 Differential Revision: D28122452 fbshipit-source-id: c076d80f8b75545f67ccc3656445ab159aaf3b46 | 30 April 2021, 20:27:15 UTC |
| 9d8d89c | Sasha Sheng | 30 April 2021, 16:58:26 UTC | [fix] pytorch lightning api change (#916) Summary: Recently, pytorch lightning internal in fbcode got updated to the recent master; My PR https://github.com/facebookresearch/mmf/issues/755 also updated OSS pytorch lightning dependency to master; This is to fix the original fix https://github.com/facebookresearch/mmf/issues/880 to use the most recent pytorch lightning api. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/916 Reviewed By: apsdehal Differential Revision: D28112996 Pulled By: ytsheng fbshipit-source-id: de462682ae0fcb98dd40c39ab5e61fe3dc77e338 | 30 April 2021, 16:59:16 UTC |
| 08f062e | Amanpreet Singh | 29 April 2021, 08:16:00 UTC | [feat] Adds audio (resnet18) and video (r2plus1d18) encoders (#879) Summary: This PR adds support for audio and video modality encoders to MMF. These can be used in conjunction with MMFTransformer. An example config has been added to showcase the usage. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/879 Test Plan: Unit tests have been added. Reviewed By: ytsheng Differential Revision: D27804875 Pulled By: apsdehal fbshipit-source-id: 9f276dab2dc711fb8e5868a029f73c16083c1782 | 29 April 2021, 08:16:39 UTC |
| 231fb16 | Sasha Sheng | 29 April 2021, 07:20:28 UTC | [feat] PL mvp1: validation + logging (#771) Summary: - [x] MVP 1. Evaluation and Logging - Eval accuracy parity between mmf_trainer and lightning_trainer - [X] Validation at the right frequency and at the end (also calculate the right metrics) - [X] Do logging (same logging format with slight difference is ok) - [x] Tests - tests only done for pytorch lightning integration - [X] test logging values are the same as `mmf_trainer`'s logging values - [X] test eval values are the same as `mmf_trainer`'s eval values - [x] test tensorboard logging the same as `mmf_trainer`'s - [ ] Blocked: - [ ] One Epoch Evaluation Run after train: [PR4945](https://github.com/PyTorchLightning/pytorch-lightning/pull/4945/files) and [PR4948](https://github.com/PyTorchLightning/pytorch-lightning/pull/4948/files) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/771 Reviewed By: simran2905 Differential Revision: D26394468 Pulled By: ytsheng fbshipit-source-id: 3fdec30efe3890248cbbc65142f02df6354e3b44 | 29 April 2021, 07:21:25 UTC |
| c671214 | Aapo Kyrola | 28 April 2021, 01:28:31 UTC | [mmf] validate batch sizes are same + debugging improvements (#902) Summary: [feat] Pull Request resolved: https://github.com/facebookresearch/mmf/pull/902 Before adding batch to reporter, sync with all workers and validate all have same batch size. If not, skip the batch. Reports in the end how many were skipped. This should add quite negligible overhead. And once the problem itself is fixed (probably by constructing the dataloader with 'drop_incomplete' -- but i am not sure where to add this), this could be removed -- but perhaps better to just keep it. In addition: - remove master's logging to a file and instead log to stderr - add CompleteInTimeOrDie() so that we get better traces than NCCL deadlocks cause. - some tiny changes. Reviewed By: apsdehal Differential Revision: D27954629 fbshipit-source-id: cf000586772f4510b717153ec66e1f9c6c269ac3 | 28 April 2021, 01:29:36 UTC |
| c29b3cd | Xingjian Shi | 26 April 2021, 19:41:50 UTC | [docs] Suggestion to try turning off build isolation for installation (#888) Summary: Issue: https://github.com/facebookresearch/mmf/issues/881 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/888 Reviewed By: ytsheng Differential Revision: D27887967 Pulled By: apsdehal fbshipit-source-id: e8bd82660e438b3b552233545483f74d45934285 | 26 April 2021, 19:42:36 UTC |
| a4e9c22 | Vaibhav Singh | 26 April 2021, 19:40:19 UTC | [fix] Fixed drop_last setting and all_gather for xla device. (#872) Summary: fixed drop_last setting for xla aligned with length computation. fixed all gather (torch.stack only needed for GPU path.) Pull Request resolved: https://github.com/facebookresearch/mmf/pull/872 Reviewed By: ytsheng Differential Revision: D27798021 Pulled By: apsdehal fbshipit-source-id: 0a80ab0fe946f3885ade9220300f6df8417b6d21 | 26 April 2021, 19:40:56 UTC |
| 62ec6cd | Jun Chen | 21 April 2021, 01:09:24 UTC | [feat][MMF] Add multi-head loss Summary: * Add multi-head loss and multi-head MLP head in MMF * Fix fblearner output metrics in train workflow * Add multi-head metric and processor for new taxonomy in MMF Commerce Reviewed By: lichengunc Differential Revision: D27071740 fbshipit-source-id: c0771bdda57f577ce2cadcd05578497132b3336c | 21 April 2021, 01:10:14 UTC |
| 1a5c2b9 | Licheng Yu | 16 April 2021, 23:45:56 UTC | [mmf_ads][feat] Multi-task Learning (#882) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/882 We fixed a small bug in `mmf/modules/losses.py`. When there are multiple loss, the returned losses from MMFLoss is a dictionary. We need to unwrap its values for summing the total loss. Internally, we add multi-task learning for mmf_ads. Specifically, now it support two tasks: 1) megataxon classification 2) img cluster classification Reviewed By: apsdehal Differential Revision: D27730656 fbshipit-source-id: bd78c8af49d7cc0a9286ee0d7dfc48ff9bade3d7 | 16 April 2021, 23:46:36 UTC |
| 5bb6f50 | Amanpreet Singh | 16 April 2021, 07:50:29 UTC | [fix] regression in tests for lightning with recent change (#880) Summary: Regression caused by https://github.com/facebookresearch/mmf/commit/868c5892cf7607cdeca3df386fa7810ca1ba9b67 Fixes https://github.com/facebookresearch/mmf/issues/876 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/880 Test Plan: pytest Reviewed By: BruceChaun Differential Revision: D27812170 Pulled By: apsdehal fbshipit-source-id: 09f1fcf9d7297c5b02bfb984ffcd330b3ebdc5e0 | 16 April 2021, 07:51:11 UTC |
| 3947693 | Vedanuj Goswami | 14 April 2021, 22:02:44 UTC | [feat] Adding movie mcan e2e model to model zoo (#867) Summary: Movie MCAN e2e model for zoo VQA2.0 TestDev Acc : 73.6 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/867 Reviewed By: ytsheng Differential Revision: D27693464 Pulled By: vedanuj fbshipit-source-id: 7e435176d0330d126ce705f91ca4277644ee7f3c | 14 April 2021, 22:03:45 UTC |
| 8855495 | Amanpreet Singh | 14 April 2021, 01:45:36 UTC | [feat,fix] CPU evaluation and report memory leak (#846) Summary: This PR has three purposes: - Allow CPU evaluation for the cases where evaluation set is big can OOM on calculation for global metrics. Enable this via evaluation.use_cpu - Fix a memory leak which is caused as first report in training loop is not detached while generating a new one. - It also fixes an issues with shallow report copy made prediction report purposes. This would cause issues in multiple metrics calculation where some metrics use prediction report and some don't. Since, it is shallow copy, first report copy shares tensors with original report which when gathered in reporter would lead to average number from all GPUs and hence that report would like into original report. This is fixed by creating a deep copy of report rather than shallow copy by introducing a copy method The PR also fixes - the incorrect config settings for mmf transformer on VQA2.0 - relaunching of a run using cmd api was adding multiple callbacks to trainer as callbacks are a class variable. This PR also reset callbacks in each setup so that multiple logs or checkpoints are seen. - remove lru caching of logging functionality as they are only called once and cause issue with caching when called in cmd api - add evaluation configuration options to trainer configs in tests Pull Request resolved: https://github.com/facebookresearch/mmf/pull/846 Test Plan: Tested with UniT, VisualBERT on Hateful Memes, MMFT on VQA2. Added extensive tests as well. Reviewed By: vedanuj, lichengunc Differential Revision: D27496441 Pulled By: apsdehal fbshipit-source-id: 3fc3243117a2068228b0a3626a029b75c5cc7d00 | 14 April 2021, 01:46:33 UTC |
| 868c589 | Ananth Subramaniam | 14 April 2021, 00:51:52 UTC | Synchronize PyTorchLightning/pytorch-lightning (revision aaccbeea@master) to github/third-party/PyTorchLightning/pytorch-lightning Reviewed By: shuyingsunshine21 Differential Revision: D27674288 fbshipit-source-id: 21af95adf952d632d10289a45fba6dd7cba6b2d9 | 14 April 2021, 00:52:48 UTC |
| 401d337 | Sasha Sheng | 13 April 2021, 02:59:39 UTC | [feat][3/N] clip text processor (#164) Summary: * Added text processor Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/164 Reviewed By: apsdehal Differential Revision: D27457434 Pulled By: ytsheng fbshipit-source-id: 85116028df1866bf0021157df6be9e8ecd69f021 | 13 April 2021, 03:00:41 UTC |
| f024b7b | Sasha Sheng | 08 April 2021, 19:39:46 UTC | [test] lightning trainer to use the new datamodule api (#860) Summary: * onboard the lightning trainer to use the new datamodule api * to test: run: - `pytest tests` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/860 Reviewed By: apsdehal Differential Revision: D27617952 Pulled By: ytsheng fbshipit-source-id: 8881ed40840b85049735bf1fdf33a3f57b0d1987 | 08 April 2021, 19:40:56 UTC |
| b093095 | dependabot[bot] | 07 April 2021, 23:01:49 UTC | Bump y18n from 3.2.1 to 3.2.2 in /website (#845) Summary: Bumps [y18n](https://github.com/yargs/y18n) from 3.2.1 to 3.2.2. <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/yargs/y18n/commits">compare view</a></li> </ul> </details> <details> <summary>Maintainer changes</summary> <p>This version was pushed to npm by <a href="https://www.npmjs.com/~oss-bot">oss-bot</a>, a new releaser for y18n since your current version.</p> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/facebookresearch/mmf/network/alerts). </details> Pull Request resolved: https://github.com/facebookresearch/mmf/pull/845 Reviewed By: ytsheng, apsdehal Differential Revision: D27495488 Pulled By: vedanuj fbshipit-source-id: b9ea33ea2e6d23d9f2c877860da5f5ce4c952516 | 07 April 2021, 23:02:47 UTC |
| 66a988a | Brett Zachary Allen | 06 April 2021, 21:44:54 UTC | [fix] Download functionality work for frcnn (#858) Summary: Fixed the downloading functionality for the FRCNN feature extraction script. This resolves the TODO of using the mmf download functionality to download the configs and models. Also updated the script to directly use the huggingface urls Resolves: https://github.com/facebookresearch/mmf/issues/836 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/858 Reviewed By: vedanuj Differential Revision: D27571075 Pulled By: brettallenyo fbshipit-source-id: ab7ff27fd2b6156946c26c619a7e67b440d8af11 | 06 April 2021, 21:45:32 UTC |
| 332f60e | Sasha Sheng | 06 April 2021, 10:58:18 UTC | [feat][2/N] clip text/image encoder (#163) Summary: * Added text/image encoders * Added the pretrained checkpoint loading for those encoders We should make it composable in the future - ie., make the encoders configurable with params Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/163 Reviewed By: vedanuj Differential Revision: D27447444 Pulled By: ytsheng fbshipit-source-id: e2de1752df497cbbe9f689ad08b482565d883e29 | 06 April 2021, 10:59:06 UTC |
| d9812a6 | Sasha Sheng | 06 April 2021, 10:52:29 UTC | [fix] test error when using the new datamodule api (#855) Summary: * Updated the TrainerTrainingLoop object to use the new datamodules API + iteration_strategy API * Fixed bug related to API change for the per_gpu_batch_size test * Fixed the test_eval test to use the new API * Fixed the default iteration strategy (constant) to use `should_exhaust_all_iterators`=True. This is when multi-task condition is false. This was the default behavior before the iteration strategy API update. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/855 Test Plan: run `pytest tests` Reviewed By: vedanuj Differential Revision: D27562857 Pulled By: ytsheng fbshipit-source-id: a9baea12b046ea2affd5f5f205224131941a40ab | 06 April 2021, 10:53:48 UTC |
| 8be4a2f | Vedanuj Goswami | 05 April 2021, 23:24:08 UTC | [chores] Upgrade to support Pytorch 1.8 (#849) Summary: Upgrading max supported version to PT 1.8.1 and torchvision 0.9.1 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/849 Reviewed By: ytsheng Differential Revision: D27543579 Pulled By: vedanuj fbshipit-source-id: 603199331a3bbf91b99ff110a1145ab3e164ba02 | 05 April 2021, 23:24:57 UTC |
| 07680c4 | Brett Zachary Allen | 05 April 2021, 22:16:51 UTC | [fix] FRCNN Feature extraction for single image (#850) Summary: Fixed single image bug that had incorrect function arguments for extracting features on a single image. Fix: https://github.com/facebookresearch/mmf/issues/848 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/850 Reviewed By: vedanuj Differential Revision: D27547598 Pulled By: brettallenyo fbshipit-source-id: 7bac9aff5521c684a048441cfee619ecb6b84b5e | 05 April 2021, 22:17:34 UTC |
| 9a7b75c | Brett Zachary Allen | 05 April 2021, 21:14:48 UTC | [feat] Inference CLI (#830) Summary: Created script to allow users to run inference using various models, text inputs, and image inputs. All that is required is using mmf_inference and passing some command line args. Here is how you use it: ``` mmf_interactive checkpoint_path=/checkpoint/brettallen/mmft/model ``` This will initiate an interactive inference script that will ask for image URL and text input to run on. It will keep prompting the user for more images and text until they say `exit`. I also added the ability to just say `same` to use the same image because I figured it was inconvenient for the user to constantly have to copy over the image URL every time they want to ask a new question. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/830 Reviewed By: apsdehal Differential Revision: D27339291 Pulled By: brettallenyo fbshipit-source-id: db190c40626229900a3530da75b61d6e6b63cdd0 | 05 April 2021, 21:15:34 UTC |
| 99b1f74 | Sasha Sheng | 05 April 2021, 00:31:02 UTC | [refactor] reporting + meter (#844) Summary: * Clean the API up to prep for moving some of the report/meter updating inside the base_model for pytorch lightning early stopping/checkpointing. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/844 Reviewed By: vedanuj Differential Revision: D27486844 Pulled By: ytsheng fbshipit-source-id: 759fbaccdc1ce2ef2a8e736fa179afeec48b89dc | 05 April 2021, 00:31:50 UTC |
| 2379d0c | Ronghang Hu | 03 April 2021, 08:52:39 UTC | [fixup] add update_registry_for_model to datamodules (#851) Summary: Call `update_registry_for_model` to datamodules to register e.g. answer num. Fix https://github.com/facebookresearch/mmf/issues/847. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/851 Test Plan: tested locally with M4C and M4C-Captioner Reviewed By: apsdehal Differential Revision: D27553712 Pulled By: ronghanghu fbshipit-source-id: 7e5a4092658e8240ad615b0601cdf40d393453fb | 03 April 2021, 08:53:24 UTC |
| 7828606 | Erjia Guan | 02 April 2021, 22:19:06 UTC | Change MagicMock from return_type to spec Summary: Unblock OSS PyTorch Diff D27327232 ``` class Dataset: pass class IterableDataset(Dataset): pass class NumbersDataset(Dataset): pass d = NumbersDataset() # Prev foo = MagicMock(return_type=c) isinstance(foo, Dataset) # False isinstance(foo, IterableDataset) # False # Now foo = MagicMock(spec=c) isinstance(foo, Dataset) # True isinstance(foo, IterableDataset) # False ``` Reviewed By: ytsheng, vedanuj, apsdehal Differential Revision: D27529078 fbshipit-source-id: 0c6fec0a7379d58dbcd6c4583dc3aed1482f28ab | 02 April 2021, 22:19:56 UTC |
| 9d7696f | Sasha Sheng | 02 April 2021, 10:32:52 UTC | [fix] Use caption only files for localized narratives (#839) Summary: vedanuj. This PR uses smaller caption files. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/839 Reviewed By: vedanuj Differential Revision: D27424834 Pulled By: ytsheng fbshipit-source-id: 3ff4a252fffa700de9e03e880dfa8cf7fadd321b | 02 April 2021, 10:33:33 UTC |
| 6f89879 | Sasha Sheng | 02 April 2021, 08:58:43 UTC | [feat] add tests for eval loop (#835) Summary: * Adds test for evaluation loop for mmf trainer * Makes it so that test folder quality is more explicit in terms of errors. I wrote this test to understand deeper on how the evaluation_loop.py works. This needs to land before the next pytorch lightning update. To test that it works run: `pytest tests` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/835 Reviewed By: vedanuj Differential Revision: D27405061 Pulled By: ytsheng fbshipit-source-id: c1c27fe824ac282d387ba37bb726cdb562f428ef | 02 April 2021, 08:59:58 UTC |
| 71554e3 | Sasha Sheng | 02 April 2021, 08:56:52 UTC | [test] reduce mem leak in tests (#837) Summary: * Fix memory leak in pytest Pull Request resolved: https://github.com/facebookresearch/mmf/pull/837 Test Plan: run `pytest tests` Reviewed By: apsdehal Differential Revision: D27421597 Pulled By: ytsheng fbshipit-source-id: 5edf023ef46fc77c82d0924c9ceb48624fd8c4e3 | 02 April 2021, 08:57:45 UTC |
| 10e3f45 | Ronghang Hu | 02 April 2021, 06:19:02 UTC | [feat] add model checkpoints on UniT w/o task embeddings (#831) Summary: Add new pretrained checkpoints of UniT w/o task embedding, which can be evaluated as follows: ``` # on a 8-GPU machine python mmf_cli/run.py \ config=projects/unit/configs/all_8_datasets/shared_dec_without_task_embed.yaml \ datasets=detection_coco,detection_visual_genome,vqa2,visual_entailment,glue_qnli,glue_sst2,glue_mnli_mismatched,glue_qqp \ model=unit \ run_type=val \ training.batch_size=8 \ checkpoint.resume_zoo=unit.all_8_datasets.shared_dec_with_coco_init_without_task_embedding ``` Results: ``` val/detection_coco/detection_mean_ap: 0.3847, val/detection_visual_genome/detection_mean_ap: 0.0331, val/vqa2/vqa_accuracy: 0.6825, val/visual_entailment/accuracy: 0.7407, val/glue_qnli/accuracy: 0.8815, val/glue_sst2/accuracy: 0.8850, val/glue_mnli_mismatched/accuracy: 0.8061, val/glue_qqp/accuracy: 0.9061 ``` Also added pretrained COCO single-task checkpoints to perform COCO initialization experiments. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/831 Test Plan: Tested locally with UniT model Reviewed By: vedanuj Differential Revision: D27528247 Pulled By: ronghanghu fbshipit-source-id: d7e4c8ec337f502b58f4c71f4b29fb6b996df03d | 02 April 2021, 06:19:49 UTC |
| f3d6f6c | Licheng Yu | 31 March 2021, 22:33:14 UTC | [mmf_ads] models trained on ads+pages (#843) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/843 MT image+text tables: - `ads_and_pages_for_xlm_processed_v5_with_xray_hash_feature:aml` - `ads_and_pages_for_xlm_processed_v5_with_xray_2048_feature:aml` with same splits as `ads_and_pages_for_xlm_processed_v5:ad_delivery`. We fix tokenizer bug in `bert_processors.py`, where the special tokens for Roberta are `<s>`, `</s>`, `<mask>`, etc, instead of Bert's `[CLS]`, `[SEP]`, `[MASK]`, etc. Besides, the padding index for roberta and bert is 1 and 0 respectively. Add `ads+pages` training and MMFT (hugginface, pytext) supports. Reviewed By: ytsheng Differential Revision: D26440611 fbshipit-source-id: 21e97162aa9f5b2363d2c68a71a20f185b7a8db6 | 31 March 2021, 22:34:09 UTC |
| 6395711 | Amanpreet Singh | 31 March 2021, 20:47:25 UTC | [fix] Reallow num_workers and pin_memory overrides from training config (#842) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/842 Recent changes for datamodules support moved the options num_workers and pin_memory along with others inside the dataset_config. This diff fixes backwards compatibility and allows specifying the num_workers and pin_memory from training config. Reviewed By: ytsheng, vedanuj, rayhou0710 Differential Revision: D27421166 fbshipit-source-id: 1fff516dcbf6a0f8bb5ab81ca6c477b53b0e4937 | 31 March 2021, 20:48:30 UTC |
| 03a0527 | Sasha Sheng | 30 March 2021, 22:24:22 UTC | [feat][1/N]: clip contrastive loss (#162) Summary: Add Contrastive Loss Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/162 Reviewed By: vedanuj Differential Revision: D27389621 Pulled By: ytsheng fbshipit-source-id: 60424e6c4cd568bdbef55d8f49cb729dc078af7b | 30 March 2021, 22:25:17 UTC |
| 29f57ea | Amanpreet Singh | 30 March 2021, 07:54:53 UTC | [refactor] Move test_reporter_config to evaluation.reporter (#838) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/838 Test reporter config as opposed to original design should be a top level config node as it is not specific to one dataset but encompasses all datasets in one go. This diff refactors that to be a top level config node so that it is not per dataset config which conflicts with overall definition of test reporter. Reviewed By: madian9 Differential Revision: D27301920 fbshipit-source-id: 6a990cd23c837fb915398a5930b9e348ff4cd626 | 30 March 2021, 07:55:36 UTC |
| a094ad7 | Amanpreet Singh | 30 March 2021, 03:39:45 UTC | [feat] Iteration strategies for multitasking (#822) Summary: This PR adds support for iteration strategies that can be used with multitasking. IterationStrategies as a concept just returns next dataset index to be sampled in each turn. The diff implements multiple of them and makes some sensible assumptions on the defaults. - Always fallback to size proportional sampling for val and test sets - Epoch based strategy only allowed in roundrobin, constant and size proportional. - Adds constant, roundrobin, constant, size proportional and ratios schemes. - constant strategy for single task runs Pull Request resolved: https://github.com/facebookresearch/mmf/pull/822 Reviewed By: vedanuj Differential Revision: D27211249 fbshipit-source-id: bce14a44730bdd830a72f4e72c350211e088c8e8 | 30 March 2021, 03:40:32 UTC |
| 542694f | Amanpreet Singh | 29 March 2021, 17:03:21 UTC | [feat] Add teardown support for datamodules (#834) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/834 Without proper teardown, the job might fail as errored and restart. This diff adds proper teardown support for datamodules which inherently will include onbox dataloader as well. Reviewed By: ytsheng, vedanuj Differential Revision: D26687357 fbshipit-source-id: 8cebc7f49055cea6906cb367b539fc4073666493 | 29 March 2021, 17:04:37 UTC |
| 12650f5 | Amanpreet Singh | 27 March 2021, 09:01:26 UTC | [feat,refactor] Support datamodules directly in dataset loader (#782) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/782 This diff refactors MMF to support datamodules directly. Reviewed By: ytsheng, vedanuj Differential Revision: D26481038 fbshipit-source-id: 6a3e36fe50a30c44f473098152e604962c2f4f8e | 27 March 2021, 09:02:20 UTC |
| 2d29089 | Jun Chen | 26 March 2021, 22:17:52 UTC | [MMF][feat] Add label smoothing cross entropy loss Summary: Add label smoothing cross entropy in loss function Differential Revision: D27092731 fbshipit-source-id: 66b6cf2bf04a2324b7d6f36d90f4a1ebb1243a35 | 26 March 2021, 22:18:55 UTC |
| e12e03d | Amanpreet Singh | 26 March 2021, 17:49:33 UTC | [fix] QoL changes to support onbox (#155) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/155 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/827 Some of the original monkey patching doesn't work with onbox dataloader inside processors. Also, sets some env variables to support distributed onbox setting. Reviewed By: ytsheng, vedanuj Differential Revision: D27306251 fbshipit-source-id: 135b320cf3a585413a8902ee6cd0f67a58ec64d7 | 26 March 2021, 17:51:02 UTC |
| 5c5c356 | Amanpreet Singh | 26 March 2021, 17:34:50 UTC | [chores] Add hateful memes config for MMF Transformer (#828) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/828 This is being used in our getting started example but is missing, so adding. Reviewed By: ytsheng, vedanuj Differential Revision: D27306253 fbshipit-source-id: 40407235e0c6feddc556efeb5af4ecda916339a8 | 26 March 2021, 17:35:52 UTC |
| e7ad822 | Shaoliang Nie | 25 March 2021, 22:49:15 UTC | [feat] Hateful Memes fine grained (#156) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/156 Pull Request resolved: https://github.com/facebookresearch/mmf/pull/829 hateful memes fine grained Reviewed By: ytsheng, apsdehal Differential Revision: D26490349 fbshipit-source-id: 670698aefd13fa0fae4d61174fe669e86ac5f7fe | 25 March 2021, 22:50:12 UTC |
| 3f1bcf4 | Amanpreet Singh | 25 March 2021, 05:11:07 UTC | [refactor] Collator samplelist conversion logic for more control (#826) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/826 This refactor allows us to use sample list conversion logic used in BatchCollator elsewhere as well. Reviewed By: vedanuj Differential Revision: D27306252 fbshipit-source-id: a9a89388e760d0b23c729f3ac43399df20fc7088 | 25 March 2021, 05:11:56 UTC |
| e88d059 | Madian Khabsa | 25 March 2021, 03:35:36 UTC | [postray] Add fblearner flows (#153) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/153 - Create flows for postray to avoid remaining dependant on faim flows - Extend path finder to search for user dir Reviewed By: rayhou0710 Differential Revision: D27040702 fbshipit-source-id: c1b166cb3b8775f25879cbe405435550478b81b7 | 25 March 2021, 03:36:40 UTC |
| 9588752 | Amanpreet Singh | 25 March 2021, 03:28:33 UTC | [feat] Inherit from PL datamodule for dataset builders (#825) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/825 After this change dataset builder will inherit from PL datamodule as at the high level both concepts are same. BaseDatasetBuilder has been adjusted to also support datamodule functionality directly. Reviewed By: ytsheng, vedanuj Differential Revision: D26505873 fbshipit-source-id: f3e1393f65bf25a2ec7161a5ef2e450c98a192e0 | 25 March 2021, 03:29:25 UTC |
| 19f3211 | Sasha Sheng | 24 March 2021, 22:07:02 UTC | [feat] add images to localized narratives (#823) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/823 Reviewed By: vedanuj Differential Revision: D27306005 Pulled By: ytsheng fbshipit-source-id: 1626f2e8762138744130fdc343d3261095293100 | 24 March 2021, 22:07:58 UTC |
| 7699ff0 | Brett Zachary Allen | 24 March 2021, 20:17:08 UTC | [feat] Inference API (#798) Summary: Added an API to run inference in these use cases: - region features -> mmft - region features -> visual_bert Based on configs, some are provided here, users can run inference on various trained models with something as simple as an image URL and a text query. Would love for people to take a look and provide feedback about my coding style and design choices. If you want to run it on your own you can pull and run ``` from mmf.utils.inference import Inference if __name__ == "__main__": inference = Inference(checkpoint_path="/checkpoint/brettallen/visualbert") answer = inference.forward( "http://i.imgur.com/1IWZX69.jpg", {"text": "what type of shoes is the woman in pink wearing"}, image_format="url", ) print(answer) ``` Pull Request resolved: https://github.com/facebookresearch/mmf/pull/798 Reviewed By: vedanuj Differential Revision: D27171794 Pulled By: brettallenyo fbshipit-source-id: 25b51e5da8cdae944b53970ce1a30b253dee339f | 24 March 2021, 20:18:15 UTC |
| 800a086 | Madian Khabsa | 24 March 2021, 03:21:27 UTC | <feat> Refactor MMF Test Reporter and add Hive Exporter (#150) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/150 - Modifies test reporter to be fetched/registered in registry - Add hive test reporter for internal support Reviewed By: apsdehal Differential Revision: D26954690 fbshipit-source-id: b8ad088d829a5353c713cb493c38f27099b43fcd | 24 March 2021, 03:23:02 UTC |
| e5edc48 | Vedanuj Goswami | 22 March 2021, 22:58:41 UTC | [fix] Fix iopath get_local_path related checkpoint issues (#817) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/817 Adds force=True for downloading manifold checkpoints so that we do not load cahced versions of the checkpoints like best.ckpt or current.ckpt Reviewed By: apsdehal Differential Revision: D27130415 fbshipit-source-id: 21546b6be43cf87bc37ecb0842fc6321b678bcbf | 22 March 2021, 23:00:03 UTC |
| b752db6 | Vedanuj Goswami | 19 March 2021, 23:50:52 UTC | [fix] Do not run validation if final val not in run_type (#816) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/816 Adding a check to final validation to run only if val is in run_type Reviewed By: ytsheng, apsdehal Differential Revision: D27130474 fbshipit-source-id: 2793a1dd3f77d4d56d737781a68973634d271157 | 19 March 2021, 23:52:20 UTC |
| 62a50f1 | Sasha Sheng | 19 March 2021, 22:29:23 UTC | [fix] tests (pytest + black) (#149) Summary: skip fb tasks on devfair pytest Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/149 Reviewed By: apsdehal Differential Revision: D27185785 Pulled By: ytsheng fbshipit-source-id: 1e88b76607fbf86a67a87ea601ce8a2c35197499 | 19 March 2021, 22:31:00 UTC |
| 3ba4725 | Amanpreet Singh | 19 March 2021, 22:17:27 UTC | [fix] length for multi dataset loader; max_epochs; eta (#820) Summary: The calculation of length for multidataloader used get_batch_size to divide total dataset length which is incorrect as get_batch_size is per device rather than global batch size. This PR fixes it and fixes snapshot interval calculation as well. This in turn: - Fixes max_epochs support - ETA - All other things dependent on this Pull Request resolved: https://github.com/facebookresearch/mmf/pull/820 Test Plan: Tested locally with MVLT runs Reviewed By: ytsheng, madian9 Differential Revision: D27184804 Pulled By: apsdehal fbshipit-source-id: d18aeb52b7e280f33c3b9e7509dafa123ed1c629 | 19 March 2021, 22:18:53 UTC |
| 08fb370 | Amanpreet Singh | 18 March 2021, 22:58:39 UTC | [refactor] Fix typings used from mmf common/typings Summary: Currently, typings used in multiple places in MMF from common/typings are incorrect. This diff fixes them. Reviewed By: ytsheng Differential Revision: D27127754 fbshipit-source-id: fccbb5df0a7ffac2f12e73a9d47e369fb6b7a4d0 | 18 March 2021, 23:00:10 UTC |
| 7fb6bdc | Amanpreet Singh | 18 March 2021, 21:24:54 UTC | [feat] Allow returning sequences in transformer encoder (#814) Summary: - For some cases such as MLM we need full sequences. It would be great if we can reuse existing transformer encoder and support that. This PR adds support for the same along with tests Pull Request resolved: https://github.com/facebookresearch/mmf/pull/814 Test Plan: Tests have been added. Tested also with MVLT Reviewed By: ytsheng, vedanuj Differential Revision: D27075345 Pulled By: apsdehal fbshipit-source-id: c8d475405d4950494831846f2eef795975e5ff6c | 18 March 2021, 21:31:54 UTC |
| bbc8035 | Amanpreet Singh | 18 March 2021, 20:15:41 UTC | [feat,fix] Make some transformer head params optional; label key for MLM (#813) Summary: This PR makes some QoL changes for transformer heads so that they can be used at places other than MMFTransformer. - Make encoder_layers and processed_sample_list options optional by default - Add label key option for mlm to allow direct fetching of maskedlabels from sample list Pull Request resolved: https://github.com/facebookresearch/mmf/pull/813 Test Plan: Tested with GUniT Reviewed By: ytsheng, vedanuj Differential Revision: D27062330 Pulled By: apsdehal fbshipit-source-id: 3f7aed573b65154960a18dffb4ca6600c0d4bb34 | 18 March 2021, 20:17:11 UTC |
| 8f9150b | Amanpreet Singh | 18 March 2021, 00:05:31 UTC | [fix] Allow running slurm scripts from directory other than mmf (#818) Summary: - Currently, hardcoded to be run only from the same directory as mmf - This PR fixes it so that it can be launched from anywhere Pull Request resolved: https://github.com/facebookresearch/mmf/pull/818 Test Plan: Tested with MVLT Reviewed By: ytsheng Differential Revision: D27136433 Pulled By: apsdehal fbshipit-source-id: cfa0cb2696a7547c1a299ee9790eaa8fd9dbd307 | 18 March 2021, 00:06:58 UTC |
| 7cc0e4e | Ananth Subramaniam | 17 March 2021, 17:47:46 UTC | [mmf] Update gradient clipping test to use almost equals for float comparison (#147) Summary: Pull Request resolved: https://github.com/fairinternal/mmf-internal/pull/147 Reviewed By: ytsheng Differential Revision: D26783077 fbshipit-source-id: 65c278e68d25736ed89c9a1669aa5f401e0548f6 | 17 March 2021, 17:49:54 UTC |
| c672d08 | Ronghang Hu | 17 March 2021, 01:30:26 UTC | [feat] add the UniT model and supporting utils (#741) Summary: This PR is built upon https://github.com/facebookresearch/mmf/pull/740 - add the UniT model - add `AdamWSkipParamsWithZeroGrad` optimizer (skips unused params) - provide dataset zoo download for VQAv2 splits w.r.t. COCO 2017. - provide model zoo download for UniT. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/741 Test Plan: Tested locally and verified the outputs --- Evaluating the UniT model zoo entry ``` # on a 8-GPU machine python mmf_cli/run.py \ config=projects/unit/configs/all_8_datasets/shared_dec.yaml \ datasets=detection_coco,detection_visual_genome,vqa2,visual_entailment,glue_qnli,glue_sst2,glue_mnli_mismatched,glue_qqp \ model=unit \ run_type=val \ training.batch_size=8 \ checkpoint.resume_zoo=unit.defaults ``` Evaluation outputs on the 8-dataset model: ``` val/detection_coco/detection_mean_ap: 0.3899, val/detection_visual_genome/detection_mean_ap: 0.0329, val/vqa2/vqa_accuracy: 0.6696, val/visual_entailment/accuracy: 0.7316, val/glue_qnli/accuracy: 0.8790, val/glue_sst2/accuracy: 0.8922, val/glue_mnli_mismatched/accuracy: 0.8090, val/glue_qqp/accuracy: 0.9065 ``` Reviewed By: vedanuj, apsdehal Differential Revision: D26894259 Pulled By: ronghanghu fbshipit-source-id: b548313f724d61c2c6e38c2cba1ef19cd735dfa9 | 17 March 2021, 01:31:59 UTC |
| f2e45f7 | Amanpreet Singh | 12 March 2021, 00:17:39 UTC | [fix,feat] Support more wider variety of user dirs (#809) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/809 This PR handles some edge cases around user dirs which weren't handle before especially in cases where python files are compiled and moved to a different place. This different place can also have compile time generated modules which are not available directly to the original folder. The diff also handles support for specifying a python file as user dir input. Lastly, this diff adds extensive test to cover and test various user dir configurations. Reviewed By: ytsheng, vedanuj Differential Revision: D26838780 fbshipit-source-id: 38e4ffaf55053df0b29ff682b26bb4ae5b6638ac | 12 March 2021, 00:18:24 UTC |
| 60c9138 | vedanuj | 11 March 2021, 11:40:07 UTC | [fix] Fix memory leak in tests (#806) Summary: Fixing some memory leak in tests. Pull Request resolved: https://github.com/facebookresearch/mmf/pull/806 Test Plan: Multiple re-runs of tests show no failure. Reviewed By: ytsheng, apsdehal Differential Revision: D26952871 Pulled By: vedanuj fbshipit-source-id: 0dc91b5f497dd234b25671c105afce788797dcb3 | 11 March 2021, 11:41:54 UTC |
| 6210494 | Madian Khabsa | 11 March 2021, 03:13:27 UTC | <fix> support providing configs that contain equal character (#808) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/808 The current parser would fail if the opt value has `=` which is needed when providing filters. This diff takes the first occurrence of `=` as the separator Reviewed By: apsdehal Differential Revision: D26955390 fbshipit-source-id: cedfb358de254a0c6d2a88744a0660984b045f47 | 11 March 2021, 03:15:10 UTC |
| c78a408 | Vedanuj Goswami | 10 March 2021, 23:41:25 UTC | [fix] Fix MMFT for loading older checkpoints, add tests and assertions (#805) Summary: Pull Request resolved: https://github.com/facebookresearch/mmf/pull/805 Fix MMFT for loading older checkpoints after head module refactor. Also adds a test to catch these errors in future. Reviewed By: ytsheng Differential Revision: D26924574 fbshipit-source-id: 09fa22b1b922a6b3d1698f0fe545d826a1bfb5c6 | 10 March 2021, 23:42:54 UTC |
