Revision a6e13ce96934354f9bae2fbe134071852e6340ee authored by Junjie Qian on 13 December 2016, 21:58:26 UTC, committed by Junjie Qian on 25 February 2017, 18:53:48 UTC
Packing aggregated matrixes into continuous buffer so less MPI operations are used. Usage: Introduce a configurable threshold size (units = KB) and only pack gradients into a continous buffer whose total size is below the threshold. This is to avoid possible memory overflow. The default threshold size is 32KB. To change this value, specify "packThresholdsizeInKB=value" (e.g. 2048 for 2MB) in train/eval config file. Notes: acquire continous buffer only if NCCL is not supported Fallback to normal execution if not enough extra continous memory available change reduce all to unblocking reduce all SimpleDistAggregate: Broadcast the aggregated headers to all nodes Configurable threshold size to pack gradients in a buffer simpleAggregator: pack the gradient into continous buffer if its size is less than PackThresholdSize Add all constant definitions to a header file in Common
1 parent 9375555
| File | Mode | Size |
|---|---|---|
| Dependencies | ||
| Documentation | ||
| Examples | ||
| Scripts | ||
| Source | ||
| Tests | ||
| Tools | ||
| Tutorials | ||
| bindings | ||
| .clang-format | -rw-r--r-- | 931 bytes |
| .gitattributes | -rw-r--r-- | 2.5 KB |
| .gitignore | -rw-r--r-- | 5.3 KB |
| .gitmodules | -rw-r--r-- | 211 bytes |
| CNTK.Common.props | -rw-r--r-- | 1.2 KB |
| CNTK.Cpp.props | -rw-r--r-- | 7.7 KB |
| CNTK.sln | -rw-r--r-- | 182.5 KB |
| CONTRIBUTING.md | -rw-r--r-- | 199 bytes |
| CppCntk.vssettings | -rw-r--r-- | 10.0 KB |
| LICENSE.md | -rw-r--r-- | 4.6 KB |
| Makefile | -rw-r--r-- | 51.3 KB |
| README.md | -rw-r--r-- | 9.5 KB |
| configure | -rwxr-xr-x | 31.9 KB |
Computing file changes ...
