sort by:
Revision Author Date Message Commit Date
535ccdb Fix docstring typo PiperOrigin-RevId: 210092924 16 October 2018, 12:57:12 UTC
c776d06 Sonnet embed: print warning about using default initializer. Eventually we will switch to using a default initializer of stddev=1. PiperOrigin-RevId: 210082974 16 October 2018, 12:57:04 UTC
a5fcdac Add clone method to snt.nets.MLP PiperOrigin-RevId: 209902324 16 October 2018, 12:56:57 UTC
257a62c Make `snt.scale_gradient` support eager mode. Much like optimizers and other tesor taking APIs, in eager mode we require users to pass us a callable which they want to scale the gradients of. >>> f = lambda x: tf.pow(x, 2) >>> f = scale_gradient(f, scale=0.1) >>> dy_scaled, tfe.gradients_function(f)(x) >>> print dy_scaled.numpy() # 0.2 PiperOrigin-RevId: 209405560 16 October 2018, 12:56:50 UTC
0e649da Implement same graph check using graph keys. tl;dr - resolves the last known issues with Sonnet and `tfe.defun`. Inside a `defun` we observe a different graph instance each time the function is traced. Sonnet asserts that each time a module is traced that it's graph has not changed. This CL changes that test to check that the graph we observe each time has the same "key" (rather than being the same instance). The key is akin to a primary key for the graph. Graph instances with the same key form part of the same parent graph (e.g. a sub-graph representing a function with key "a" shares variables with a regular tf.Graph with key "a" which it is a part of). DifferentGraphError used to fire undesirably in defun in the following cases (both of which are fixed in this cl): 1) `enter_variable_scope` is used in conjunction with `_build` (e.g. in the constructor or via `snt.reuse_variables`. 2) `defun` re-traces `_build` due to the input signature changing (e.g. Tensor shape changing, or Python parameter values changing). PiperOrigin-RevId: 209131983 16 October 2018, 12:56:44 UTC
6a42160 Pass optional named arguments to the wrapped sonnet module. Named arguments can be used to change the behavior of sonnet modules. For example, it's not uncommon to use dropout to regularize a deep neural network. However, dropout should only be used during the training of the network, not afterwards. PiperOrigin-RevId: 208786599 16 October 2018, 12:56:36 UTC
1fb3f4e Make MLP/ConvNet compatible with `tfe.defun`. PiperOrigin-RevId: 208621259 16 October 2018, 12:56:29 UTC
0452bbd Make use of `variable_creator_scope` for variable tracking. Variable creators are stackable factory functions used to control variable creation. By placing a variable creator at the top of the stack we can observe all variables being created or re-requested (e.g. via `tf.get_variable`) and store them in `self._all_variables`. Additionally this works for the case where a `custom_getter` creates more than one variable (as in the "Bayes by Backprop" case). PiperOrigin-RevId: 208034949 16 October 2018, 12:56:22 UTC
1f310fd Simplify module stacks by removing weakref to graph. We perform same-graph checks agressively when the module is connected, so it seems to me there is no simple way to end up with multiple-graphs in the module stack. The reason I'd like to remove this is that it causes some weirdness with `defun`, since if two modules `defun` themselves you get two different capturing graphs which would cause the module stack functionality to break down: >>> @tfe.defun ... def foo(): ... print 'foo', id(tf.get_default_graph()) ... return bar() >>> @tfe.defun ... def bar(): ... print 'bar', id(tf.get_default_graph()) ... return tf.ones([]) >>> id(tf.get_default_graph()) 140336549583184 >>> foo(); foo 140336571240336 bar 140336555261456 There are other issues with composing defuns which mean it's not simple to add a Sonnet specific test for this (yet!), however we have many tests which cover the functionality enabled by the module stack (pushing variables from child to parent) and those still pass :) PiperOrigin-RevId: 207307133 16 October 2018, 12:56:15 UTC
34fd32d Add count_variables_by_type() to Sonnet. PiperOrigin-RevId: 207275694 16 October 2018, 12:56:08 UTC
ebaf4a4 Add variable properties to Sonnet modules. These methods query `self._all_variables` and don't filter using global collections. Their name and contract matches many other TF objects (e.g. defun's `PolymorphicFunction`, `Template` and Keras `Model`). A nice benefit of this is that you can use defun and access the variables from your module in a consistent way: >>> mod = snt.Linear(1) >>> if FLAGS.use_defun: ... mod = tfe.defun(mod) >>> mod(inputs); >>> mod.variables (<tf.Variable 'linear/b:0' shape=(1,) dtype=float32_ref>, <tf.Variable 'linear/w:0' shape=(1, 1) dtype=float32_ref>) Additionally we follow the more intuitive behaviour of `get_all_variables` so modules like Sequential don't return the empty collection: >>> mod = snt.Sequential([snt.Linear(1)]) >>> mod(tf.constant([[1.0]]); >>> mod.variables (<tf.Variable 'linear/b:0' shape=(1,) dtype=float32_ref>, <tf.Variable 'linear/w:0' shape=(1, 1) dtype=float32_ref>) PiperOrigin-RevId: 207253894 16 October 2018, 12:55:59 UTC
f2f8d85 Sonnet version update produced on Tuesday, 31. July 2018 PiperOrigin-RevId: 206595149 31 July 2018, 15:45:47 UTC
f2f0812 Add dense/sparse gradient option to the Sonnet Embed module. Adds an option to densify the gradients into a tensor instead of passing indexed-slices. This is beneficial in the backward pass when gradients are passed to the parameter servers, and can speed up performance for moderately sized embeddings. PiperOrigin-RevId: 206166675 31 July 2018, 15:40:26 UTC
dca79d7 Keep track of last connected output size in DeepRNN. PiperOrigin-RevId: 205828972 31 July 2018, 15:40:15 UTC
2bf3452 Compatibility with recent tf variable changes. PiperOrigin-RevId: 204941059 31 July 2018, 15:40:03 UTC
44ba888 Facility to register for notifications of module being connected to the graph. PiperOrigin-RevId: 204450050 17 July 2018, 15:57:48 UTC
2b4d757 Make snt.LSTMBlockCell an actual class for the benefit of code that instantiates using module_name/class_name. PiperOrigin-RevId: 204102372 17 July 2018, 15:57:42 UTC
798e8e0 Internal change PiperOrigin-RevId: 203951281 17 July 2018, 15:57:36 UTC
f7c759c Run Sonnet RNN tests in eager and graph mode. PiperOrigin-RevId: 203924472 17 July 2018, 15:57:30 UTC
52129f6 Don't track connected subgraphs in eager mode. In eager mode subgraphs are connected several orders of magnitude more times than is typical in graph mode. Keeping track of all connected subgraphs means we hold onto all input/output tensors and end up quite quickly OOM-ing. PiperOrigin-RevId: 203894783 17 July 2018, 15:57:19 UTC
5568c56 Run Sonnet base tests in eager and graph. PiperOrigin-RevId: 203483937 17 July 2018, 15:57:13 UTC
8215c83 Use semantic_version to check that correct version of TF is installed PiperOrigin-RevId: 203111217 17 July 2018, 15:57:07 UTC
719cd49 Expose RNNCellWrapper and wrap_rnn_cell_class PiperOrigin-RevId: 202639317 17 July 2018, 15:57:01 UTC
c2f607c Allow a TensorFlow RNNCell to be wrapped as an RNNCore. PiperOrigin-RevId: 202477539 17 July 2018, 15:56:52 UTC
e15eb74 Apply bias only once in ConvLSTM. https://github.com/deepmind/sonnet/issues/65 PiperOrigin-RevId: 202473272 17 July 2018, 15:56:42 UTC
9672140 Extend Conv1DLSTM and Conv2DLSTM to support Layer Norm. PiperOrigin-RevId: 202471664 17 July 2018, 15:56:35 UTC
56b40d6 Internal change. PiperOrigin-RevId: 202470337 17 July 2018, 15:56:29 UTC
7a8f633 Run Sonnet basic tests in eager and graph mode. PiperOrigin-RevId: 202335246 17 July 2018, 15:56:20 UTC
dc7dd5c Custom getter that overrides specific named parameters. PiperOrigin-RevId: 202308055 17 July 2018, 15:56:14 UTC
86147a0 Keep signature of original method when using snt.reuse_variables. PiperOrigin-RevId: 202124746 17 July 2018, 15:56:08 UTC
62386f9 Use `get_shape` instead of `shape` for convolution variables. If one of convolution's variables is a PartitionedVariable, then it has no `shape` attribute. However, both Variables and PartitionedVariables have a `get_shape()` function. Use this instead. Modify the shared conv tests to catch this. PiperOrigin-RevId: 201985043 17 July 2018, 15:56:02 UTC
bc08dbf Refactor of VectorQuantizerEMA supporting return of encoding_indices The CL also allows to recover quantized codes from input encoding_indices. PiperOrigin-RevId: 201949176 17 July 2018, 15:55:56 UTC
cc8db83 Re-enable Sonnet net tests in eager mode. PiperOrigin-RevId: 201929983 17 July 2018, 15:55:50 UTC
1175778 Internal change. PiperOrigin-RevId: 201527233 17 July 2018, 15:55:44 UTC
4e5741a Temporarily remove eager/graph annotation until the class level one is released. PiperOrigin-RevId: 201509633 17 July 2018, 15:55:38 UTC
6926503 Add a bidirectional recurrent core to sonnet. Based off encoder component from: https://arxiv.org/pdf/1409.0473.pdf I avoided tf.while_loop as it complicates the implementation and we want explicit access to states and outputs from all steps of the sequence. PiperOrigin-RevId: 200346579 17 July 2018, 15:55:32 UTC
4f34f4f Support for tf.bfloat16 in Sonnet. PiperOrigin-RevId: 200045112 17 July 2018, 15:55:26 UTC
e61955b Add eager mode tests for sonnet nets. PiperOrigin-RevId: 199800734 17 July 2018, 15:55:20 UTC
ef9146d Fix dependencies in examples/BUILD - SciPy was missing. PiperOrigin-RevId: 199638219 17 July 2018, 15:55:01 UTC
1eac5c6 Make dataset_nth_farthest python3 compatible. PiperOrigin-RevId: 199635116 17 July 2018, 15:54:55 UTC
5332938 Make rmc_nth_farthest python3 compatible PiperOrigin-RevId: 199633253 17 July 2018, 15:54:49 UTC
220832e Fix rmc_nth_farthest.ipynb PiperOrigin-RevId: 199629407 17 July 2018, 15:54:43 UTC
9fe0a35 Adds demo for Relational Memory Core for "nth farthest" task from paper. PiperOrigin-RevId: 199622123 17 July 2018, 15:54:30 UTC
31bc79c Sonnet version update produced on Tuesday, 05. June 2018 PiperOrigin-RevId: 199312224 05 June 2018, 16:53:02 UTC
0c9ed73 Updated changelog for version 1.23 PiperOrigin-RevId: 199311026 05 June 2018, 16:45:51 UTC
05e2109 Fix error message in DeepRNN. Because we enumerate through core_sizes[1:] we need to add 1 to i when printing out which core we are referring to in the case of an error. Also fixed an issue where one of the shapes printed had the first dimension removed, and the other one didn't. PiperOrigin-RevId: 199127178 05 June 2018, 16:45:43 UTC
3dd3d33 Make relational_memory python3 compatible. PiperOrigin-RevId: 199124668 05 June 2018, 16:45:35 UTC
3ab2bea Change test assertion function. PiperOrigin-RevId: 199121362 05 June 2018, 16:45:26 UTC
5b314d6 Add relational memory module to sonnet. Adding relational memory implementation from "Relational Recurrent Neural Networks", Santoro et al., 2018. PiperOrigin-RevId: 199116117 05 June 2018, 16:45:17 UTC
b66dadc Sonnet version update produced on Friday, 01. June 2018 PiperOrigin-RevId: 198867845 01 June 2018, 14:20:59 UTC
71c16c1 Switch to using private member variables when possible. There can be subtle differences to the public and private versions of member variables in the conv module (eg, Convolution's `stride` is the conv size + 2 but `_stride` is the conv size). The current module mixes and matches these. To reduce cognitive load a bit, we switch to using the private member variable when possible. PiperOrigin-RevId: 198857690 01 June 2018, 14:20:52 UTC
3ec0d90 Additional changes to VQVAE notebook to make it python3 compatible. PiperOrigin-RevId: 198735118 01 June 2018, 10:48:37 UTC
ce869c1 Make brnn_ptb_test write checkpoints to temp directory PiperOrigin-RevId: 198729764 01 June 2018, 10:48:31 UTC
f6a930b MergeDims: also handle dimensions of size zero. PiperOrigin-RevId: 198699387 01 June 2018, 10:48:25 UTC
6307466 Use six to load cPickle in VQVAE notebook PiperOrigin-RevId: 198694898 01 June 2018, 10:48:18 UTC
35778ef Updated changelog for Sonnet version 1.21 PiperOrigin-RevId: 198559887 30 May 2018, 14:59:18 UTC
d6a02aa internal change PiperOrigin-RevId: 198559691 30 May 2018, 14:59:11 UTC
45d0f76 Jupyter Notebook demonstrating VQ-VAE training on CIFAR-10. PiperOrigin-RevId: 198556907 30 May 2018, 14:59:05 UTC
024f263 Add control dependencie to VectorQuantizerEMA to avoid potential non-deterministic results. PiperOrigin-RevId: 197884089 30 May 2018, 14:43:34 UTC
f72c4a5 Improve Sonnet's MergeDim behaviour on partially defined shapes. PiperOrigin-RevId: 197863174 30 May 2018, 14:43:27 UTC
c3d3bc2 Add support for custom getters in ConvNet2D PiperOrigin-RevId: 197546042 30 May 2018, 14:43:20 UTC
8a7ca0a Add SeparableConv1D class to sonnet. PiperOrigin-RevId: 197408513 30 May 2018, 14:43:14 UTC
85492de skip_connection deprecation warning is printed only once PiperOrigin-RevId: 197370137 30 May 2018, 14:43:07 UTC
0277e53 Sonnet version update produced on Tuesday, 08. May 2018 PiperOrigin-RevId: 195825275 30 May 2018, 14:42:59 UTC
c4ac2b1 Add copyright header PiperOrigin-RevId: 195399161 30 May 2018, 14:42:46 UTC
c595084 Add VQ-VAE plus EMA variant to Sonnet. PiperOrigin-RevId: 195398448 30 May 2018, 14:42:36 UTC
f34755c Add snt.summarize_variables to Sonnet. This prints a summary of #variables, #scalars and memory usage for each datatype. PiperOrigin-RevId: 194779974 08 May 2018, 15:49:30 UTC
df245c0 Make an error more explicit when the Sonnet module name is not a string. PiperOrigin-RevId: 194254264 08 May 2018, 15:49:14 UTC
c838ebf Sonnet version update produced on Tuesday, 24. April 2018 PiperOrigin-RevId: 194098244 24 April 2018, 16:28:00 UTC
6db202c Make brnn_ptb and ptb_reader python3 compatible. Fixes https://github.com/deepmind/sonnet/issues/79 PiperOrigin-RevId: 194091717 24 April 2018, 15:51:37 UTC
50b41df Update installation instructions PiperOrigin-RevId: 194085445 24 April 2018, 15:51:30 UTC
a30da96 Add a unit test for brnn_ptb. Runs a small model with fake data for 1 epoch. PiperOrigin-RevId: 194079346 24 April 2018, 15:51:20 UTC
5c7e355 Merge SeparableConv2D into _ConvND Merge the final convolution class into _ConvND. We also add a property field for `channel_multiplier` in `DepthwiseConv2D` as well as `SeparableConv2D`. PiperOrigin-RevId: 193007472 24 April 2018, 15:51:11 UTC
1a313f6 Sonnet version update produced on Tuesday, 10. April 2018 PiperOrigin-RevId: 192293232 10 April 2018, 16:01:18 UTC
a86f044 Remove skip_connnection option from ConvLSTM. PiperOrigin-RevId: 192131304 10 April 2018, 14:10:07 UTC
600268f Update test PiperOrigin-RevId: 192129236 10 April 2018, 14:10:01 UTC
5980cf9 Move CausalConv related functionality into subclass. There are a few bits of functionality that are only related to CausalConv1D and don't need to be in _ConvND. We're moving them into the subclass and calling _ConvND machinery for the rest. PiperOrigin-RevId: 192117666 10 April 2018, 14:09:56 UTC
1ca6866 Refactor DepthwiseConv2D to use _ConvND. Rewrite DepthwiseConv2D so that it shares as much functionality with _ConvND as possible. PiperOrigin-RevId: 192116657 10 April 2018, 14:09:50 UTC
23cb8d9 Refactor InPlaneConv2D to use _ConvND This refactor removes almost all of the code in InPlaneConv2D and uses _ConvND instead. We also fix some tests that were failing when run on the GPU in conv_gpu_test due to numpy & tensorflow version upgrades. PiperOrigin-RevId: 192109587 10 April 2018, 14:09:45 UTC
9802714 Add dilation rate tests for ConvNet2D. Adding testing for the `rates` argument in ConvNet2D. PiperOrigin-RevId: 191893167 10 April 2018, 14:09:39 UTC
159adc5 Remove spurious spaces in doc. PiperOrigin-RevId: 190449915 10 April 2018, 14:09:34 UTC
e660931 Use tensorflow nest for nest operations. PiperOrigin-RevId: 188871826 10 April 2018, 14:09:28 UTC
1bee593 Use nest from pytflib to get rid of deprecation messages. PiperOrigin-RevId: 188862864 10 April 2018, 14:09:21 UTC
4596fd0 Sonnet version update produced on Monday, 12. March 2018 PiperOrigin-RevId: 188861040 13 March 2018, 12:50:56 UTC
80d4784 Fix typo in DeepRNN docs. PiperOrigin-RevId: 188474053 12 March 2018, 14:10:13 UTC
e331172 Support the root scope (an empty string) in get_variable_scope. PiperOrigin-RevId: 188341067 12 March 2018, 14:10:08 UTC
7d65b8a Improve the way reuse_variables handles name scopes This fixes a bug where outer name scopes were ignored by reuse_variables. PiperOrigin-RevId: 188163153 12 March 2018, 14:10:03 UTC
b1191ad Docstring clarification PiperOrigin-RevId: 187880593 12 March 2018, 14:09:58 UTC
e62f660 Add mention to get_all_variables in snt.Sequential warning. PiperOrigin-RevId: 187456723 12 March 2018, 14:09:53 UTC
553159b Added optional dilation rates argument for ConvNet2D. PiperOrigin-RevId: 186779967 12 March 2018, 14:09:48 UTC
c855672 Sort variables returned by get_all_variables() by name. The variables returned by get_all_variables() are stored in a set, so they must be sorted to determine an ordering over them. This change sorts the variables by name before returning them. PiperOrigin-RevId: 186752578 12 March 2018, 14:09:42 UTC
a1d1a37 Add missing argument to example code. PiperOrigin-RevId: 186613534 12 March 2018, 14:09:36 UTC
108fce7 Bump version to 1.17 PiperOrigin-RevId: 186463459 21 February 2018, 17:32:51 UTC
c522ae6 Updated README - TensorFlow v1.5 required GIT_ORIGIN_REV_ID=264bc35ad75f1dad605407295e857846237917be PiperOrigin-RevId: 185666441 21 February 2018, 16:10:30 UTC
62f4399 Implementation of get_all_variables() for sonnet modules. We introduce a module call stack, which tracks the order in which modules are called. When a module enters __call__ (or _enter_variable_scope) it adds itself to the top of the stack. Variables created inside of the custom_getter are added to a collection specific to the module on the top of the stack. Before exiting __call__ (or _enter_variable_scope) the module moves all variables added to this graph collection into `_all_variables`, removes itself from the top of the stack, and adds all of the variables from `self._all_variables` to collection for the module that is currently at the top of the module stack. PiperOrigin-RevId: 185664981 21 February 2018, 16:10:23 UTC
06b8e4f Replace keep_dims with keepdims in call to tf.reduce_prod() PiperOrigin-RevId: 185524194 21 February 2018, 16:10:17 UTC
f9e3859 Fix an error message which was incorrect. PiperOrigin-RevId: 185509698 21 February 2018, 16:10:10 UTC
c75bca4 Fix comment. PiperOrigin-RevId: 185501790 21 February 2018, 16:10:03 UTC
0737482 Internal change. PiperOrigin-RevId: 185389700 21 February 2018, 16:09:55 UTC
551d652 Fix Python3 incompatibilities in new tests and methods. PiperOrigin-RevId: 183831798 30 January 2018, 16:10:36 UTC
back to top