| 176d8b3 | Afroz Mohiuddin | 21 November 2019, 18:21:15 UTC | Revert "Fix decoding in prepend mode (#1726)" This reverts commit c825d126643d1c8864d43d828df0e7a868eaa180. | 21 November 2019, 18:21:15 UTC |
| d381f2b | Prasasto Adi | 21 November 2019, 17:58:28 UTC | En-Id untokenized parallel corpora (#1733) Change to untokenized parallel corpora due to t2t automatically tokenizes sentences. | 21 November 2019, 17:58:28 UTC |
| c825d12 | Seppo Enarvi | 21 November 2019, 17:58:13 UTC | Fix decoding in prepend mode (#1726) * Create an integer problem_0_steps variable. * Save inputs to the feature "partial_targets" when prepend_mode is not "none". * Removed a second call to update_hparams_for_universal_transformer(). Fixes hyperparameter sets universal_transformer_big and universal_transformer_base_tpu. * Fix a bug to make partial targets work for beam size > 1 The dimension of the multiplication of the partial targets was wrong: (a, b, c, d) --> (a, b, c, d, a, b, c, d) Correct multiplication needs to be: (a, b, c, d) --> (a, a, b, b, c, c, d, d) This is because it is (batch_size * beam_size) instead of (beam_size * batch_size). Basically, tf.tile needs to be replaced by tf.repeat which is introduced in tf 1.15. This is a workaround for tf 1.14. | 21 November 2019, 17:58:13 UTC |
| 67ddb40 | Sepehr Sameni | 21 November 2019, 17:57:51 UTC | use batch_size in _test_img2img_transformer (#1724) | 21 November 2019, 17:57:51 UTC |
| b3610a2 | Afroz Mohiuddin | 21 November 2019, 05:16:23 UTC | - Bump up T2T version to 1.15.0 in lockstep with TF 1.15 - Add tf_slim as a requirement, for tf.contrib.slim replacement. - Delete `attacks` and `pruning_strategies` they are aliases of `attack` and `pruning_strategy` respectively. PiperOrigin-RevId: 281670284 | 21 November 2019, 05:27:03 UTC |
| db00074 | Mohammad Taghi Saffar | 19 November 2019, 01:15:20 UTC | Fix features for decoding on TPUs. PiperOrigin-RevId: 281188677 | 19 November 2019, 01:16:00 UTC |
| f7d93b7 | Lukasz Kaiser | 18 November 2019, 02:41:35 UTC | More work on Trax imports: add optimizers and Trainer. PiperOrigin-RevId: 280977657 | 18 November 2019, 02:42:32 UTC |
| 4d96546 | T2T Team | 11 November 2019, 21:46:00 UTC | Enabled encoder-decoder attention for the multi-encoder case. PiperOrigin-RevId: 279807999 | 11 November 2019, 21:49:53 UTC |
| 8c0736a | Marc van Zee | 05 November 2019, 17:04:51 UTC | Fix bug in universal transformer hyperparameter range. PiperOrigin-RevId: 278638337 | 05 November 2019, 17:05:31 UTC |
| b679a88 | T2T Team | 04 November 2019, 20:19:42 UTC | Prevent float-type mismatches by converting predictions to float64. (The labels had already been so converted.) PiperOrigin-RevId: 278437095 | 04 November 2019, 20:20:30 UTC |
| 9ed57fa | T2T Team | 04 November 2019, 17:17:25 UTC | Add metrics for TWO_CLASS_ACCURACY, TWO_CLASS_LOG_LIKELIHOOD, and UNPADDED_MSE. PiperOrigin-RevId: 278394549 | 04 November 2019, 17:18:06 UTC |
| 2eebc98 | Jacob Burnim | 02 November 2019, 02:57:58 UTC | Update deprecated calls of tfp.distributions.* to tfp.math*. PiperOrigin-RevId: 278049446 | 02 November 2019, 02:58:37 UTC |
| 06bc1eb | Daniel De Freitas Adiwardana | 01 November 2019, 21:32:27 UTC | Enabling t2t_decoder to read hparams from output_dir/hparams.json when available. PiperOrigin-RevId: 278002517 | 01 November 2019, 21:33:07 UTC |
| d636032 | T2T Team | 01 November 2019, 21:23:29 UTC | Adding TPU and tiny hparams sets for evolved_transformer PiperOrigin-RevId: 278000866 | 01 November 2019, 21:24:02 UTC |
| 5185a40 | T2T Team | 01 November 2019, 13:58:14 UTC | Fix typo. PiperOrigin-RevId: 277919398 | 01 November 2019, 13:58:56 UTC |
| bcc4306 | T2T Team | 31 October 2019, 17:58:10 UTC | Fix deprecation warning. PiperOrigin-RevId: 277757850 | 31 October 2019, 17:59:03 UTC |
| ab918e0 | Blake Hechtman | 29 October 2019, 16:49:50 UTC | [TENSOR2TENSOR] Use an einsum instead of matmul. PiperOrigin-RevId: 277303581 | 29 October 2019, 16:50:28 UTC |
| dc71240 | T2T Team | 28 October 2019, 01:48:28 UTC | Implementation of Neural Assistant model. PiperOrigin-RevId: 276982866 | 28 October 2019, 01:49:10 UTC |
| 687fbe0 | T2T Team | 25 October 2019, 22:51:04 UTC | Fix deprecation warning. PiperOrigin-RevId: 276776475 | 25 October 2019, 22:51:41 UTC |
| f5d8c3e | T2T Team | 25 October 2019, 14:55:58 UTC | Fix deprecation warning. PiperOrigin-RevId: 276687096 | 25 October 2019, 14:56:34 UTC |
| adb27c7 | Kristy Choi | 24 October 2019, 17:47:26 UTC | Minor changes to tensor2tensor to handle cases where inputs and targets share the same symbol modality. PiperOrigin-RevId: 276520696 | 24 October 2019, 17:48:11 UTC |
| ae8dc45 | Ian Simon | 23 October 2019, 23:44:59 UTC | fix shape assertion that causes relative attention to fail unnecessarily PiperOrigin-RevId: 276378348 | 23 October 2019, 23:45:43 UTC |
| e4d9992 | T2T Team | 23 October 2019, 21:56:16 UTC | Fix PY3 compatibility bug with map() input. PiperOrigin-RevId: 276357497 | 23 October 2019, 21:56:56 UTC |
| 1547c25 | T2T Team | 22 October 2019, 15:46:58 UTC | Fix deprecation warning. PiperOrigin-RevId: 276072314 | 22 October 2019, 15:47:56 UTC |
| 6b0193c | T2T Team | 18 October 2019, 15:28:34 UTC | Fix deprecation warning. PiperOrigin-RevId: 275476920 | 18 October 2019, 15:29:14 UTC |
| 2330203 | T2T Team | 18 October 2019, 07:46:57 UTC | Fix deprecation warning. PiperOrigin-RevId: 275421785 | 18 October 2019, 07:47:38 UTC |
| e6da46c | Sergei Lebedev | 17 October 2019, 21:43:29 UTC | Fixed cases where tf.TensorShape was constructed with float dimensions This is a prerequisite for making TensorShape and Dimension more strict about the types of their arguments. PiperOrigin-RevId: 275342516 | 17 October 2019, 21:55:22 UTC |
| be35054 | T2T Team | 16 October 2019, 19:26:18 UTC | Update t2t example notebooks to Python 3 PiperOrigin-RevId: 275088373 | 16 October 2019, 19:26:58 UTC |
| 3aca2ab | David So | 14 October 2019, 21:08:39 UTC | Add random seed for test consistency. PiperOrigin-RevId: 274656425 | 14 October 2019, 21:09:20 UTC |
| b5d8381 | Gaurav Jain | 12 October 2019, 08:24:25 UTC | Rename internal_convert_to_tensor for performance Calling ops.internal_convert_to_tensor is more efficient than calling ops.convert_to_tensor due to skipping the deprecated_argument_lookup and also less python function calling overhead. We thus swap these functions names so we can optimize most code paths. PiperOrigin-RevId: 274321742 | 12 October 2019, 08:25:22 UTC |
| 42b35dd | Mohammad Babaeizadeh | 11 October 2019, 20:35:45 UTC | No round sampling for L2 models. This prevents the model to predict negative numbers. PiperOrigin-RevId: 274238535 | 11 October 2019, 20:36:26 UTC |
| c395bfd | Piotr Kozakowski | 08 October 2019, 22:23:37 UTC | Correct the dependencies for the MBRL notebook. PiperOrigin-RevId: 273618262 | 08 October 2019, 22:24:32 UTC |
| 8428263 | Derek Murray | 07 October 2019, 22:00:12 UTC | Internal change PiperOrigin-RevId: 273383958 | 07 October 2019, 22:00:51 UTC |
| 6c7c601 | Afroz Mohiuddin | 05 October 2019, 19:36:19 UTC | Move Trax out of T2T into its own repo. PiperOrigin-RevId: 273078371 | 05 October 2019, 19:36:56 UTC |
| a8024e8 | Lukasz Kaiser | 04 October 2019, 19:44:42 UTC | Update Position Lookup Transformer config to use recent Trax additions. PiperOrigin-RevId: 272937269 | 04 October 2019, 19:45:22 UTC |
| e9d6b10 | T2T Team | 04 October 2019, 16:00:34 UTC | General progress on the first section (Layers) of the intro notebook. PiperOrigin-RevId: 272886830 | 04 October 2019, 16:01:26 UTC |
| 4ea0ef7 | Piotr Kozakowski | 04 October 2019, 02:16:38 UTC | Switch {MemoryEfficient -> TimeBin}CausalAttention and add a config for SimPLe with a serialized policy. PiperOrigin-RevId: 272791159 | 04 October 2019, 02:17:13 UTC |
| e258b4d | Piotr Kozakowski | 04 October 2019, 00:17:48 UTC | Update the world model -> policy parameter initialization code after the change making loss functions layers. PiperOrigin-RevId: 272775707 | 04 October 2019, 00:18:32 UTC |
| ec5c72e | Lukasz Kaiser | 04 October 2019, 00:16:18 UTC | Allow to factorize hash creation to lower hashing complexity. PiperOrigin-RevId: 272775435 | 04 October 2019, 00:16:54 UTC |
| e73eb74 | Piotr Kozakowski | 04 October 2019, 00:14:39 UTC | Don't write summaries by default in OnlineTuneEnv. This takes up a lot of space and we can generate those metrics in the evaluator afterwards. PiperOrigin-RevId: 272775151 | 04 October 2019, 00:15:36 UTC |
| 232225f | Afroz Mohiuddin | 03 October 2019, 19:36:55 UTC | Execute `jupyter nbconvert` on python3 instead of python2. python2 fails with: `RuntimeError: Kernel died before replying to kernel_info` python3 seems to pass. PiperOrigin-RevId: 272718077 | 03 October 2019, 19:37:40 UTC |
| 176148c | T2T Team | 03 October 2019, 18:42:15 UTC | Fix attention rng mismatch between forward and reverse direction PiperOrigin-RevId: 272707157 | 03 October 2019, 18:42:56 UTC |
| 9f29518 | T2T Team | 03 October 2019, 17:07:39 UTC | Remove unused params arg from MulConstant. PiperOrigin-RevId: 272685588 | 03 October 2019, 17:08:23 UTC |
| 8e23892 | Afroz Mohiuddin | 03 October 2019, 05:27:49 UTC | Fix deselect syntax :/ PiperOrigin-RevId: 272590689 | 03 October 2019, 05:28:32 UTC |
| 5913ad6 | Afroz Mohiuddin | 03 October 2019, 02:33:51 UTC | Disable DatasetPacking test (in Travis) till we can figure out why autograph errors out. https://travis-ci.org/afrozenator/tensor2tensor/jobs/592856608 PiperOrigin-RevId: 272572935 | 03 October 2019, 02:35:20 UTC |
| d7d65f1 | Afroz Mohiuddin | 03 October 2019, 02:33:41 UTC | Pin tfp to 0.7.0 -- their latest version 0.8.0 pins cloudpickle to 1.1.1, which gym doesn't like (it needs ~1.2) Also upgrade pip/setuptools in travis, this gives better errors so I was able to pin this down. PiperOrigin-RevId: 272572911 | 03 October 2019, 02:34:25 UTC |
| 1e5e7b4 | Afroz Mohiuddin | 03 October 2019, 02:07:00 UTC | Move the tf.autograph.to_graph annotation just before using the _scan_step_fn. Intended to fix this - https://travis-ci.org/tensorflow/tensor2tensor/jobs/587720856 I can't figure out why this became a problem suddenly. PiperOrigin-RevId: 272570396 | 03 October 2019, 02:08:34 UTC |
| a2fff1c | T2T Team | 03 October 2019, 02:06:56 UTC | Repurpose demo notebook as a practical introduction to Trax layers. PiperOrigin-RevId: 272570383 | 03 October 2019, 02:07:41 UTC |
| 1fa0b46 | Afroz Mohiuddin | 03 October 2019, 01:54:10 UTC | Bump up tf version in setup.py, missed this the last time. PiperOrigin-RevId: 272568653 | 03 October 2019, 01:54:42 UTC |
| 5b34091 | pb | 03 October 2019, 01:28:47 UTC | Merge of PR #1720 PiperOrigin-RevId: 272565130 | 03 October 2019, 01:29:31 UTC |
| 8be915a | pb | 03 October 2019, 01:07:45 UTC | Deep_discriminator - missed relu (#1720) | 03 October 2019, 01:07:45 UTC |
| 1843c72 | Piotr Kozakowski | 03 October 2019, 00:18:10 UTC | Implement fast inference for TimeBinCausalAttention. Also added a test for consistency between {DotProduct,TimeBin}CausalAttention and added a bunch of comments. PiperOrigin-RevId: 272554499 | 03 October 2019, 00:18:50 UTC |
| c9ba7ec | Piotr Kozakowski | 02 October 2019, 18:51:46 UTC | Update mask_id in the remaining configs. PiperOrigin-RevId: 272486903 | 02 October 2019, 18:52:20 UTC |
| dbff3ac | T2T Team | 02 October 2019, 18:31:51 UTC | Internal PiperOrigin-RevId: 272482483 | 02 October 2019, 18:32:34 UTC |
| 2df12d5 | Piotr Kozakowski | 02 October 2019, 01:19:47 UTC | Add a bin_len parametrization to TimeBinCausalAttention and add padding to support variable-length sequences. Both parametrizations are allowed, so no existing configs need changing. PiperOrigin-RevId: 272339876 | 02 October 2019, 01:20:22 UTC |
| 76872f6 | Shawn Simister | 02 October 2019, 00:24:26 UTC | Extend Neural Stack to support Deque by reading and writing in both directions. PiperOrigin-RevId: 272331503 | 02 October 2019, 00:25:12 UTC |
| 71e10f3 | Piotr Kozakowski | 01 October 2019, 23:56:45 UTC | Fix a corner case in BoxSpaceSerializer, causing a bug in SimPLe training data generation. PiperOrigin-RevId: 272326328 | 01 October 2019, 23:57:23 UTC |
| 7c2012c | Afroz Mohiuddin | 01 October 2019, 22:03:54 UTC | Delete grpcio from setup.py, since we deleted the gRPC code. PiperOrigin-RevId: 272303379 | 01 October 2019, 22:06:19 UTC |
| 6e4cd23 | Lukasz Kaiser | 01 October 2019, 20:19:17 UTC | Make losses and metrics instances of layers in Trax. PiperOrigin-RevId: 272279474 | 01 October 2019, 20:20:00 UTC |
| d7ac88f | T2T Team | 01 October 2019, 18:18:04 UTC | Internal PiperOrigin-RevId: 272253582 | 01 October 2019, 18:18:42 UTC |
| 67bdc3e | Piotr Kozakowski | 01 October 2019, 17:47:33 UTC | Back to passing around state in the world model. Inference works now. PiperOrigin-RevId: 272245726 | 01 October 2019, 17:48:12 UTC |
| 6f7a221 | T2T Team | 01 October 2019, 17:28:51 UTC | Revise docstring for base Layer class. PiperOrigin-RevId: 272240953 | 01 October 2019, 17:29:29 UTC |
| bf33311 | Piotr Kozakowski | 01 October 2019, 02:42:04 UTC | Rename batch_fun to batch_fn in OnlineTune baselines. PiperOrigin-RevId: 272120070 | 01 October 2019, 02:42:41 UTC |
| 6222d0c | Peng Wang | 01 October 2019, 00:12:46 UTC | Added a centralized switch in tf-numpy to disable/enable float64. PiperOrigin-RevId: 272100203 | 01 October 2019, 00:21:04 UTC |
| f3f018f | T2T Team | 01 October 2019, 00:07:53 UTC | Don't sort inside TimeBinCausalAttention PiperOrigin-RevId: 272099375 | 01 October 2019, 00:20:10 UTC |
| 331d4fb | Afroz Mohiuddin | 30 September 2019, 22:23:32 UTC | Rollback of the sync-RL pipeline, we'll use async going forward. PiperOrigin-RevId: 272078120 | 30 September 2019, 22:24:08 UTC |
| 3867955 | T2T Team | 30 September 2019, 20:56:16 UTC | Adjust relative masking strengths PiperOrigin-RevId: 272058835 | 30 September 2019, 20:56:59 UTC |
| 58f6969 | Piotr Kozakowski | 30 September 2019, 20:49:19 UTC | Fix a bug arising from not feeding the action symbols to the model during decoding. PiperOrigin-RevId: 272057330 | 30 September 2019, 20:50:04 UTC |
| 08a851c | T2T Team | 30 September 2019, 19:35:30 UTC | Rename classes used in reformer PiperOrigin-RevId: 272041361 | 30 September 2019, 19:36:11 UTC |
| 949e985 | T2T Team | 30 September 2019, 19:04:59 UTC | Unify configs for reformer experiments PiperOrigin-RevId: 272035162 | 30 September 2019, 19:05:41 UTC |
| bc8f167 | Piotr Kozakowski | 30 September 2019, 18:55:21 UTC | Update/add OnlineTune config files. PiperOrigin-RevId: 272032842 | 30 September 2019, 18:56:01 UTC |
| b8c4998 | T2T Team | 30 September 2019, 18:36:13 UTC | Rename batch_fun to batch_fn, to align with other ..._fn names. PiperOrigin-RevId: 272028547 | 30 September 2019, 18:37:01 UTC |
| 019b6be | T2T Team | 30 September 2019, 06:42:55 UTC | Internal code clean-ups, around initialization and parameters. PiperOrigin-RevId: 271916998 | 30 September 2019, 06:43:40 UTC |
| 2f35350 | RJ Skerry-Ryan | 29 September 2019, 21:38:14 UTC | Rewrite references to tf.contrib.signal to tf.signal. PiperOrigin-RevId: 271871580 | 29 September 2019, 21:38:54 UTC |
| 694b00a | Lukasz Kaiser | 29 September 2019, 14:44:12 UTC | Add test for Transformer-Revnet. PiperOrigin-RevId: 271844734 | 29 September 2019, 14:44:52 UTC |
| 8c41386 | T2T Team | 29 September 2019, 01:12:31 UTC | Change Layer.__call__ to return results rather than (results, state). PiperOrigin-RevId: 271788568 | 29 September 2019, 01:13:04 UTC |
| 5f359ee | Pierre Ruyssen | 28 September 2019, 19:28:55 UTC | Internal change PiperOrigin-RevId: 271766307 | 28 September 2019, 19:29:35 UTC |
| beb485c | T2T Team | 28 September 2019, 14:16:04 UTC | Give Layer.__call__ option to take params & state from object itself. PiperOrigin-RevId: 271745879 | 28 September 2019, 14:16:48 UTC |
| e2254bc | T2T Team | 28 September 2019, 01:10:31 UTC | Change names of key attributes of base Layer class. PiperOrigin-RevId: 271685286 | 28 September 2019, 01:11:15 UTC |
| 2e56e66 | Piotr Kozakowski | 28 September 2019, 01:02:31 UTC | Undo rescaling the observations to the [-1, 1] interval. Instead rescale the controls to the observation range. PiperOrigin-RevId: 271684386 | 28 September 2019, 01:03:18 UTC |
| 03bac1a | Piotr Kozakowski | 28 September 2019, 00:23:05 UTC | Allow overriding model keyword arguments for inference in SimulatedEnvProblem. PiperOrigin-RevId: 271679587 | 28 September 2019, 00:23:39 UTC |
| 87e0f62 | Piotr Kozakowski | 28 September 2019, 00:18:35 UTC | Pass the correct number of controls to the policy network in PolicySchedule. PiperOrigin-RevId: 271678993 | 28 September 2019, 00:19:14 UTC |
| b9c5129 | Piotr Kozakowski | 27 September 2019, 22:21:48 UTC | Update the config for SimPLe and solve name conflicts using gin scopes. PiperOrigin-RevId: 271658940 | 27 September 2019, 22:22:29 UTC |
| 40bdf03 | Thang Luong | 27 September 2019, 22:02:42 UTC | Add a unit test for attention_bias_local. PiperOrigin-RevId: 271655306 | 27 September 2019, 22:03:25 UTC |
| df87f90 | Piotr Kozakowski | 27 September 2019, 21:58:26 UTC | Zero out all observations if we get any NaNs. Now we are NaN-proof! PiperOrigin-RevId: 271654300 | 27 September 2019, 21:59:57 UTC |
| a80c332 | Piotr Kozakowski | 27 September 2019, 21:58:14 UTC | Cache the initial trajectories in memory to save time on loading them. PiperOrigin-RevId: 271654265 | 27 September 2019, 21:58:55 UTC |
| e29cadd | Piotr Kozakowski | 27 September 2019, 21:20:27 UTC | Initialize the policy trainer in SimPLe lazily. PiperOrigin-RevId: 271646475 | 27 September 2019, 21:21:00 UTC |
| 5ba0a4e | Piotr Kozakowski | 27 September 2019, 20:57:49 UTC | Run PPO optimization on minibatches, so we can use heavier policy networks. PiperOrigin-RevId: 271641725 | 27 September 2019, 20:58:32 UTC |
| c290b16 | T2T Team | 27 September 2019, 16:17:14 UTC | Rename trainer-internal function to clarify its intent. PiperOrigin-RevId: 271584428 | 27 September 2019, 16:18:05 UTC |
| 05f222d | Lukasz Kaiser | 26 September 2019, 18:13:57 UTC | Adjust learning rate of ResNet to correct for the change in Momentum optimizer. PiperOrigin-RevId: 271390165 | 26 September 2019, 18:14:41 UTC |
| 049b9d8 | Lukasz Kaiser | 25 September 2019, 18:46:57 UTC | Update Transformer copy config to make it easier to play with settings. PiperOrigin-RevId: 271171680 | 25 September 2019, 18:47:51 UTC |
| b1c4370 | T2T Team | 25 September 2019, 18:12:39 UTC | Clarify Layer.__call__ by extracting a method for custom gradients. PiperOrigin-RevId: 271163869 | 25 September 2019, 18:13:27 UTC |
| 9dff225 | Lukasz Kaiser | 25 September 2019, 07:57:05 UTC | Add mixture of Gaussian PDF computations (both diagonal and full). PiperOrigin-RevId: 271071908 | 25 September 2019, 07:57:45 UTC |
| 4fce0b7 | Piotr Kozakowski | 24 September 2019, 18:20:16 UTC | Update PolicySchedule to control arbitrary nontrainable parameters. PiperOrigin-RevId: 270947079 | 24 September 2019, 18:21:03 UTC |
| 1632111 | Shawn Simister | 22 September 2019, 23:23:07 UTC | Refactoring to use named tuples in the controller to make it easier to read. Also refactoring Neural Stack cell so that it can be extended to a double ended queue later on. PiperOrigin-RevId: 270571957 | 22 September 2019, 23:23:45 UTC |
| 0333819 | Piotr Kozakowski | 22 September 2019, 02:14:23 UTC | Implement fast inference for TransformerLM. Also updated the simulated environment to use it. PiperOrigin-RevId: 270491153 | 22 September 2019, 02:14:55 UTC |
| 7a7c067 | Lukasz Kaiser | 22 September 2019, 01:13:35 UTC | Use hashed attention by default in configs, 128 kv size, add sweeps for nhashes and a large sweep. PiperOrigin-RevId: 270487144 | 22 September 2019, 01:14:14 UTC |
| 4343cfe | T2T Team | 21 September 2019, 23:15:05 UTC | Stash hash buckets in reversible transformer PiperOrigin-RevId: 270479770 | 21 September 2019, 23:15:45 UTC |
| 44da274 | Lukasz Kaiser | 21 September 2019, 00:48:58 UTC | Configs for reformer experiments and one more tie-in. PiperOrigin-RevId: 270385869 | 21 September 2019, 00:49:44 UTC |
| 7484887 | T2T Team | 20 September 2019, 23:23:29 UTC | Add base Reformer config for enwik8 PiperOrigin-RevId: 270373209 | 20 September 2019, 23:24:08 UTC |
