https://github.com/deeplearning4j/dl4j-examples
Tip revision: bf53c259f6ba09f10e8fb03def6d1c797cb84f7b authored by Shams Ul Azeem on 24 November 2017, 06:31:35 UTC
Updated: Determining cloud cover notebook + added ipynb format
Updated: Determining cloud cover notebook + added ipynb format
Tip revision: bf53c25
01. MultiLayerNetwork and ComputationGraph.json
{"paragraphs":[{"text":"%md\n### Note\n\nView the README.md [here](https://github.com/deeplearning4j/dl4j-examples/tree/overhaul_tutorials/tutorials/README.md) to learn about installing, setting up dependencies and importing notebooks in Zeppelin","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Note</h3>\n<p>View the README.md <a href=\"https://github.com/deeplearning4j/dl4j-examples/tree/overhaul_tutorials/tutorials/README.md\">here</a> to learn about installing, setting up dependencies and importing notebooks in Zeppelin</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714026_1118687120","id":"20171012-032354_1839085744","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:5316","user":"anonymous","dateFinished":"2017-10-19T02:47:21+0000","dateStarted":"2017-10-19T02:47:21+0000"},{"title":"MultiLayerNetwork And ComputationGraph","text":"%md\n\n### DL4j Network Architectures\n---\n\nDL4j provides the following network architectures:\n1. 'MultiLayerNetwork'\n2. 'ComputationGraph'\n\n\n### 1. MultiLayerNetwork\n'MultiLayerNetwork' consists of a single input layer and a single output layer with a stack of layers in between them.\n\n### 2. ComputationGraph\n'ComputationGraph' is used for constructing networks with a more complex architecture than 'MultiLayerNetwork'. \nIt can have multiple input layers, multiple output layers and the layers in between can be connected through a direct acyclic graph.\n\n---\n\n### Network Configurations\nWhether you create 'MultiLayerNetwork' or 'ComputationGraph', you have to provide a network configuration to it through 'NeuralNetConfiguration.Builder'.\n'NeuralNetConfiguration.Builder', as the name tells, provides a Builder pattern to configure a network architecture.\nFor Creating 'MultiLayerNetwork' we build a 'MultiLayerConfiguraion' and for 'ComputationGraph' it's 'ComputationGraphConfiguration'\n\nThe pattern goes like this: [High Level Configuration] -> [Configure Layers] -> [Pretraining and Backprop Configuration] -> [Build Configuration]","dateUpdated":"2017-10-19T02:38:34+0000","config":{"lineNumbers":false,"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>DL4j Network Architectures</h3>\n<hr/>\n<p>DL4j provides the following network architectures:<br/>1. ‘MultiLayerNetwork’<br/>2. ‘ComputationGraph’</p>\n<h3>1. MultiLayerNetwork</h3>\n<p>‘MultiLayerNetwork’ consists of a single input layer and a single output layer with a stack of layers in between them.</p>\n<h3>2. ComputationGraph</h3>\n<p>‘ComputationGraph’ is used for constructing networks with a more complex architecture than ‘MultiLayerNetwork’.<br/>It can have multiple input layers, multiple output layers and the layers in between can be connected through a direct acyclic graph.</p>\n<hr/>\n<h3>Network Configurations</h3>\n<p>Whether you create ‘MultiLayerNetwork’ or ‘ComputationGraph’, you have to provide a network configuration to it through ‘NeuralNetConfiguration.Builder’.<br/>‘NeuralNetConfiguration.Builder’, as the name tells, provides a Builder pattern to configure a network architecture.<br/>For Creating ‘MultiLayerNetwork’ we build a ‘MultiLayerConfiguraion’ and for ‘ComputationGraph’ it’s ‘ComputationGraphConfiguration’</p>\n<p>The pattern goes like this: [High Level Configuration] -> [Configure Layers] -> [Pretraining and Backprop Configuration] -> [Build Configuration]</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714029_-673359317","id":"20171004-170427_1364811928","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5317","user":"anonymous","dateFinished":"2017-10-19T02:47:21+0000","dateStarted":"2017-10-19T02:47:21+0000"},{"text":"%md\n\n### Required imports","dateUpdated":"2017-10-19T02:48:15+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Required imports</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714029_13712091","id":"20171012-033908_1315178710","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5318","user":"anonymous","dateFinished":"2017-10-19T02:48:15+0000","dateStarted":"2017-10-19T02:48:15+0000"},{"title":"","text":"import org.deeplearning4j.nn.api.OptimizationAlgorithm\r\nimport org.deeplearning4j.nn.conf.graph.MergeVertex\r\nimport org.deeplearning4j.nn.conf.layers.{DenseLayer, GravesLSTM, OutputLayer, RnnOutputLayer}\r\nimport org.deeplearning4j.nn.conf.{ComputationGraphConfiguration, MultiLayerConfiguration, NeuralNetConfiguration}\r\nimport org.deeplearning4j.nn.graph.ComputationGraph\r\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\r\nimport org.deeplearning4j.nn.weights.WeightInit\r\nimport org.nd4j.linalg.activations.Activation\r\nimport org.nd4j.linalg.learning.config.Nesterovs\r\nimport org.nd4j.linalg.lossfunctions.LossFunctions","user":"anonymous","dateUpdated":"2017-10-19T02:47:08+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"import org.deeplearning4j.nn.api.OptimizationAlgorithm\nimport org.deeplearning4j.nn.conf.graph.MergeVertex\nimport org.deeplearning4j.nn.conf.layers.{DenseLayer, GravesLSTM, OutputLayer, RnnOutputLayer}\nimport org.deeplearning4j.nn.conf.{ComputationGraphConfiguration, MultiLayerConfiguration, NeuralNetConfiguration}\nimport org.deeplearning4j.nn.graph.ComputationGraph\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.learning.config.Nesterovs\nimport org.nd4j.linalg.lossfunctions.LossFunctions\n"}]},"apps":[],"jobName":"paragraph_1508380714032_1928162405","id":"20171005-000419_678819715","dateCreated":"2017-10-19T02:38:34+0000","dateStarted":"2017-10-19T02:47:21+0000","dateFinished":"2017-10-19T02:47:24+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5319"},{"text":"%md\n\n### Building a MultiLayerConfiguration","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Building a MultiLayerConfiguration</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714033_-4853491","id":"20171012-033951_284959928","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5320","user":"anonymous","dateFinished":"2017-10-19T02:47:22+0000","dateStarted":"2017-10-19T02:47:22+0000"},{"title":"","text":"val multiLayerConf: MultiLayerConfiguration = new NeuralNetConfiguration.Builder()\r\n .seed(123).learningRate(0.1).iterations(1).optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT).updater(new Nesterovs(0.9)) //High Level Configuration\r\n .list() //For configuring MultiLayerNetwork we call the list method\r\n .layer(0, new DenseLayer.Builder().nIn(784).nOut(100).weightInit(WeightInit.XAVIER).activation(Activation.RELU).build()) //Configuring Layers\r\n .layer(1, new OutputLayer.Builder().nIn(100).nOut(10).weightInit(WeightInit.XAVIER).activation(Activation.SIGMOID).build())\r\n .pretrain(false).backprop(true) //Pretraining and Backprop Configuration\r\n .build() //Building Configuration","dateUpdated":"2017-10-19T02:48:19+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"multiLayerConf: org.deeplearning4j.nn.conf.MultiLayerConfiguration =\n{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"confs\" : [ {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : {\n \"dense\" : {\n \"activationFn\" : {\n \"ReLU\" : { }\n },\n \"adamMeanDecay\" : \"NaN\",\n \"adamVarDecay\" : \"NaN\",\n \"biasInit\" : 0.0,\n \"biasLearningRate\" : 0.1,\n \"dist\" : null,\n \"dropOut\" : 0.0,\n \"epsilon\" : \"NaN\",\n \"gradientNormalization\" : \"None\",\n \"gradientNormalizationThreshold\" : 1.0,\n \"iupdater\" : {\n \"@class\" : \"org.nd4j.linalg.learning.config.Nesterovs\",\n \"learningRate\" : 0.1,\n \"momentum\" : 0.9\n ..."}]},"apps":[],"jobName":"paragraph_1508380714033_1853637692","id":"20171005-001318_1291178273","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5321","user":"anonymous","dateFinished":"2017-10-19T02:47:24+0000","dateStarted":"2017-10-19T02:47:22+0000"},{"title":"","text":"%md \r\n\r\n### What we did here?\r\n---\r\n\r\n**- High Level Configuration**\r\n\r\nFunction | Details\r\n---------------- | -------------\r\nseed | For keeping the network outputs reproducable during runs by initializing weights and other network randomizations through a seed\r\nlearningRate | For identifying the network learning rate\r\niterations | For identifying the number of optimization iterations\r\noptimizationAlgo | Optimization Algorithm to use for training. Run 'OptimizationAlgorithm.values().foreach { println }' to see different optimization algorithms that you can use.\r\nupdater | Algorithm to be used for updating the parameters\r\n\r\n---\r\n**- Configuration of Layers**\r\n\r\nHere we are calling list() to get the 'ListBuilder'. It provides us the necessary api to add layers to the network through the 'layer(arg1, arg2)' function.\r\n- The first parameter is the index of the position where the layer needs to be added.\r\n- The second parameter is the type of layer we need to add to the network.\r\n\r\nTo build and add a layer we use a similar builder pattern as:\r\n\r\nFunction | Details\r\n---------------- | -------------\r\nnIn | The number of inputs coming from the previous layer. (In the first layer, it represents the input it is going to take from the input layer)\r\nnOut | The number of outputs it's going to send to the next layer. (For output layer it represents the labels here)\r\nweightInit | The type of weights initialization to use for the layer parameters. Run 'WeightInit.values().foreach { println }' to see different weight initializations that you can use.\r\nactivation | The activation function between layers. Run 'Activation.values().foreach { println }' to see different activations that you can use.\r\n\r\n---\r\n**- Pretraining and Backprop Configuration**\r\n\r\nFunction | Details\r\n---------------- | -------------\r\npretrain | False if training from scratch\r\nbackprop | Whether to backprop or not\r\n\r\n---\r\n**- Building a Graph**\r\n\r\nFinally, the last build() call builds the configuration for us\r\n","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>What we did here?</h3>\n<hr/>\n<p><strong>- High Level Configuration</strong></p>\n<table>\n <thead>\n <tr>\n <th>Function </th>\n <th>Details</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td>seed </td>\n <td>For keeping the network outputs reproducable during runs by initializing weights and other network randomizations through a seed</td>\n </tr>\n <tr>\n <td>learningRate </td>\n <td>For identifying the network learning rate</td>\n </tr>\n <tr>\n <td>iterations </td>\n <td>For identifying the number of optimization iterations</td>\n </tr>\n <tr>\n <td>optimizationAlgo </td>\n <td>Optimization Algorithm to use for training. Run ‘OptimizationAlgorithm.values().foreach { println }’ to see different optimization algorithms that you can use.</td>\n </tr>\n <tr>\n <td>updater </td>\n <td>Algorithm to be used for updating the parameters</td>\n </tr>\n </tbody>\n</table>\n<hr/>\n<p><strong>- Configuration of Layers</strong></p>\n<p>Here we are calling list() to get the ‘ListBuilder’. It provides us the necessary api to add layers to the network through the ‘layer(arg1, arg2)’ function.<br/>- The first parameter is the index of the position where the layer needs to be added.<br/>- The second parameter is the type of layer we need to add to the network.</p>\n<p>To build and add a layer we use a similar builder pattern as:</p>\n<table>\n <thead>\n <tr>\n <th>Function </th>\n <th>Details</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td>nIn </td>\n <td>The number of inputs coming from the previous layer. (In the first layer, it represents the input it is going to take from the input layer)</td>\n </tr>\n <tr>\n <td>nOut </td>\n <td>The number of outputs it’s going to send to the next layer. (For output layer it represents the labels here)</td>\n </tr>\n <tr>\n <td>weightInit </td>\n <td>The type of weights initialization to use for the layer parameters. Run ‘WeightInit.values().foreach { println }’ to see different weight initializations that you can use.</td>\n </tr>\n <tr>\n <td>activation </td>\n <td>The activation function between layers. Run ‘Activation.values().foreach { println }’ to see different activations that you can use.</td>\n </tr>\n </tbody>\n</table>\n<hr/>\n<p><strong>- Pretraining and Backprop Configuration</strong></p>\n<table>\n <thead>\n <tr>\n <th>Function </th>\n <th>Details</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td>pretrain </td>\n <td>False if training from scratch</td>\n </tr>\n <tr>\n <td>backprop </td>\n <td>Whether to backprop or not</td>\n </tr>\n </tbody>\n</table>\n<hr/>\n<p><strong>- Building a Graph</strong></p>\n<p>Finally, the last build() call builds the configuration for us</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714035_-119901548","id":"20171006-025852_1017122274","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5322","user":"anonymous","dateFinished":"2017-10-19T02:47:25+0000","dateStarted":"2017-10-19T02:47:25+0000"},{"title":"","text":"%md\n### Reality checking for our MultiLayerConfiguration\nYou can get your network configuration as String, JSON or YAML for reality checking.\nFor JSON we can use the 'toJson()' function","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Reality checking for our MultiLayerConfiguration</h3>\n<p>You can get your network configuration as String, JSON or YAML for reality checking.<br/>For JSON we can use the ‘toJson()’ function</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714036_1522038628","id":"20171005-001735_1522259409","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5323","user":"anonymous","dateFinished":"2017-10-19T02:47:25+0000","dateStarted":"2017-10-19T02:47:25+0000"},{"text":"println(multiLayerConf.toJson)","dateUpdated":"2017-10-19T02:48:24+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"confs\" : [ {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : {\n \"dense\" : {\n \"activationFn\" : {\n \"ReLU\" : { }\n },\n \"adamMeanDecay\" : \"NaN\",\n \"adamVarDecay\" : \"NaN\",\n \"biasInit\" : 0.0,\n \"biasLearningRate\" : 0.1,\n \"dist\" : null,\n \"dropOut\" : 0.0,\n \"epsilon\" : \"NaN\",\n \"gradientNormalization\" : \"None\",\n \"gradientNormalizationThreshold\" : 1.0,\n \"iupdater\" : {\n \"@class\" : \"org.nd4j.linalg.learning.config.Nesterovs\",\n \"learningRate\" : 0.1,\n \"momentum\" : 0.9\n },\n \"l1\" : 0.0,\n \"l1Bias\" : 0.0,\n \"l2\" : 0.0,\n \"l2Bias\" : 0.0,\n \"layerName\" : \"layer0\",\n \"learningRate\" : 0.1,\n \"learningRateSchedule\" : null,\n \"momentum\" : 0.9,\n \"momentumSchedule\" : { },\n \"nin\" : 784,\n \"nout\" : 100,\n \"rho\" : \"NaN\",\n \"rmsDecay\" : \"NaN\",\n \"updater\" : \"NESTEROVS\",\n \"weightInit\" : \"XAVIER\"\n }\n },\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"variables\" : [ ]\n }, {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : {\n \"output\" : {\n \"activationFn\" : {\n \"Sigmoid\" : { }\n },\n \"adamMeanDecay\" : \"NaN\",\n \"adamVarDecay\" : \"NaN\",\n \"biasInit\" : 0.0,\n \"biasLearningRate\" : 0.1,\n \"dist\" : null,\n \"dropOut\" : 0.0,\n \"epsilon\" : \"NaN\",\n \"gradientNormalization\" : \"None\",\n \"gradientNormalizationThreshold\" : 1.0,\n \"iupdater\" : {\n \"@class\" : \"org.nd4j.linalg.learning.config.Nesterovs\",\n \"learningRate\" : 0.1,\n \"momentum\" : 0.9\n },\n \"l1\" : 0.0,\n \"l1Bias\" : 0.0,\n \"l2\" : 0.0,\n \"l2Bias\" : 0.0,\n \"layerName\" : \"layer1\",\n \"learningRate\" : 0.1,\n \"learningRateSchedule\" : null,\n \"lossFn\" : {\n \"MCXENT\" : {\n \"softmaxClipEps\" : 1.0E-10\n }\n },\n \"lossFunction\" : \"MCXENT\",\n \"momentum\" : 0.9,\n \"momentumSchedule\" : { },\n \"nin\" : 100,\n \"nout\" : 10,\n \"rho\" : \"NaN\",\n \"rmsDecay\" : \"NaN\",\n \"updater\" : \"NESTEROVS\",\n \"weightInit\" : \"XAVIER\"\n }\n },\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"variables\" : [ ]\n } ],\n \"inferenceWorkspaceMode\" : \"SEPARATE\",\n \"inputPreProcessors\" : { },\n \"iterationCount\" : 0,\n \"pretrain\" : false,\n \"tbpttBackLength\" : 20,\n \"tbpttFwdLength\" : 20,\n \"trainingWorkspaceMode\" : \"NONE\"\n}\n"}]},"apps":[],"jobName":"paragraph_1508380714036_-754353283","id":"20171006-032859_1092583294","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5324","user":"anonymous","dateFinished":"2017-10-19T02:47:25+0000","dateStarted":"2017-10-19T02:47:25+0000"},{"title":"","text":"%md\n### Creating a MultiLayerNetwork\n\nFinally, to create a 'MultiLayerNetwork', we pass the configuration to it as shown below","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Creating a MultiLayerNetwork</h3>\n<p>Finally, to create a ‘MultiLayerNetwork’, we pass the configuration to it as shown below</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714037_1168882479","id":"20171006-033536_821667427","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5325","user":"anonymous","dateFinished":"2017-10-19T02:47:25+0000","dateStarted":"2017-10-19T02:47:25+0000"},{"text":"val multiLayerNetwork : MultiLayerNetwork = new MultiLayerNetwork(multiLayerConf)","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"multiLayerNetwork: org.deeplearning4j.nn.multilayer.MultiLayerNetwork = org.deeplearning4j.nn.multilayer.MultiLayerNetwork@be851bc\n"}]},"apps":[],"jobName":"paragraph_1508380714038_1550318478","id":"20171006-033602_2145744030","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5326","user":"anonymous","dateFinished":"2017-10-19T02:47:26+0000","dateStarted":"2017-10-19T02:47:25+0000"},{"text":"%md\n### Building a ComputationGraphConfiguration","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Building a ComputationGraphConfiguration</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714041_-1480156476","id":"20171012-034316_76709280","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5327","user":"anonymous","dateFinished":"2017-10-19T02:47:26+0000","dateStarted":"2017-10-19T02:47:26+0000"},{"title":"","text":"val computationGraphConf : ComputationGraphConfiguration = new NeuralNetConfiguration.Builder()\r\n .seed(123).learningRate(0.1).iterations(1).optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT).updater(new Nesterovs(0.9)) //High Level Configuration\r\n .graphBuilder() //For configuring ComputationGraph we call the graphBuilder method\r\n .addInputs(\"input\") //Configuring Layers\r\n .addLayer(\"L1\", new DenseLayer.Builder().nIn(3).nOut(4).build(), \"input\")\r\n .addLayer(\"out1\", new OutputLayer.Builder().lossFunction(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD).nIn(4).nOut(3).build(), \"L1\")\r\n .addLayer(\"out2\", new OutputLayer.Builder().lossFunction(LossFunctions.LossFunction.MSE).nIn(4).nOut(2).build(), \"L1\")\r\n .setOutputs(\"out1\",\"out2\")\r\n .pretrain(false).backprop(true) //Pretraining and Backprop Configuration\r\n .build() //Building configuration","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"computationGraphConf: org.deeplearning4j.nn.conf.ComputationGraphConfiguration =\n{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"defaultConfiguration\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : null,\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"v..."}]},"apps":[],"jobName":"paragraph_1508380714042_-1193233194","id":"20171005-001910_831598009","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5328","user":"anonymous","dateFinished":"2017-10-19T02:47:27+0000","dateStarted":"2017-10-19T02:47:26+0000"},{"title":"","text":"%md\n\n### What we did here?\n---\n\nThe only difference here is the way we are building layers.\nInstead of calling the 'list()' function, we call the 'graphBuilder()' to get a 'GraphBuilder' for building our 'ComputationGraphConfiguration'\nFollowing table explains what each function of a 'GraphBuilder' does\n\n---\n\nFunction | Details\n---------------- | -------------\naddInputs | A list of strings telling the network what layers to use as input layers\naddLayer | First parameter is the layer name, then the layer object and finally a list of strings defined previously to feed this layer as inputs\nsetOutputs | A list of strings telling the network what layers to use as output layers\n\n---\n\nThe output layers defined here use another function 'lossFunction' to define what loss function to use. \nUse LossFunctions.LossFunction.values().foreach { println } to see what loss functions are available.","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>What we did here?</h3>\n<hr/>\n<p>The only difference here is the way we are building layers.<br/>Instead of calling the ‘list()’ function, we call the ‘graphBuilder()’ to get a ‘GraphBuilder’ for building our ‘ComputationGraphConfiguration’<br/>Following table explains what each function of a ‘GraphBuilder’ does</p>\n<hr/>\n<table>\n <thead>\n <tr>\n <th>Function </th>\n <th>Details</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td>addInputs </td>\n <td>A list of strings telling the network what layers to use as input layers</td>\n </tr>\n <tr>\n <td>addLayer </td>\n <td>First parameter is the layer name, then the layer object and finally a list of strings defined previously to feed this layer as inputs</td>\n </tr>\n <tr>\n <td>setOutputs </td>\n <td>A list of strings telling the network what layers to use as output layers</td>\n </tr>\n </tbody>\n</table>\n<hr/>\n<p>The output layers defined here use another function ‘lossFunction’ to define what loss function to use.<br/>Use LossFunctions.LossFunction.values().foreach { println } to see what loss functions are available.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714042_-580599386","id":"20171005-003455_853806","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5329","user":"anonymous","dateFinished":"2017-10-19T02:47:27+0000","dateStarted":"2017-10-19T02:47:27+0000"},{"title":"","text":"%md\n### Reality checking for our ComputationGraphConfiguration\nYou can get your network configuration as String, JSON or YAML for reality checking.\nFor JSON we can use the 'toJson()' function","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Reality checking for our ComputationGraphConfiguration</h3>\n<p>You can get your network configuration as String, JSON or YAML for reality checking.<br/>For JSON we can use the ‘toJson()’ function</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714044_-1082797100","id":"20171006-040529_1681699382","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5330","user":"anonymous","dateFinished":"2017-10-19T02:47:27+0000","dateStarted":"2017-10-19T02:47:27+0000"},{"text":"println(computationGraphConf.toJson)","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"defaultConfiguration\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : null,\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"variables\" : [ ]\n },\n \"inferenceWorkspaceMode\" : \"SEPARATE\",\n \"iterationCount\" : 0,\n \"networkInputs\" : [ \"input\" ],\n \"networkOutputs\" : [ \"out1\", \"out2\" ],\n \"pretrain\" : false,\n \"tbpttBackLength\" : 20,\n \"tbpttFwdLength\" : 20,\n \"trainingWorkspaceMode\" : \"NONE\",\n \"vertexInputs\" : {\n \"L1\" : [ \"input\" ],\n \"out1\" : [ \"L1\" ],\n \"out2\" : [ \"L1\" ]\n },\n \"vertices\" : {\n \"L1\" : {\n \"LayerVertex\" : {\n \"layerConf\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : {\n \"dense\" : {\n \"activationFn\" : {\n \"Sigmoid\" : { }\n },\n \"adamMeanDecay\" : \"NaN\",\n \"adamVarDecay\" : \"NaN\",\n \"biasInit\" : 0.0,\n \"biasLearningRate\" : 0.1,\n \"dist\" : null,\n \"dropOut\" : 0.0,\n \"epsilon\" : \"NaN\",\n \"gradientNormalization\" : \"None\",\n \"gradientNormalizationThreshold\" : 1.0,\n \"iupdater\" : {\n \"@class\" : \"org.nd4j.linalg.learning.config.Nesterovs\",\n \"learningRate\" : 0.1,\n \"momentum\" : 0.9\n },\n \"l1\" : 0.0,\n \"l1Bias\" : 0.0,\n \"l2\" : 0.0,\n \"l2Bias\" : 0.0,\n \"layerName\" : \"L1\",\n \"learningRate\" : 0.1,\n \"learningRateSchedule\" : null,\n \"momentum\" : 0.9,\n \"momentumSchedule\" : { },\n \"nin\" : 3,\n \"nout\" : 4,\n \"rho\" : \"NaN\",\n \"rmsDecay\" : \"NaN\",\n \"updater\" : \"NESTEROVS\",\n \"weightInit\" : \"XAVIER\"\n }\n },\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"variables\" : [ ]\n },\n \"outputVertex\" : false,\n \"preProcessor\" : null\n }\n },\n \"out1\" : {\n \"LayerVertex\" : {\n \"layerConf\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : {\n \"output\" : {\n \"activationFn\" : {\n \"Sigmoid\" : { }\n },\n \"adamMeanDecay\" : \"NaN\",\n \"adamVarDecay\" : \"NaN\",\n \"biasInit\" : 0.0,\n \"biasLearningRate\" : 0.1,\n \"dist\" : null,\n \"dropOut\" : 0.0,\n \"epsilon\" : \"NaN\",\n \"gradientNormalization\" : \"None\",\n \"gradientNormalizationThreshold\" : 1.0,\n \"iupdater\" : {\n \"@class\" : \"org.nd4j.linalg.learning.config.Nesterovs\",\n \"learningRate\" : 0.1,\n \"momentum\" : 0.9\n },\n \"l1\" : 0.0,\n \"l1Bias\" : 0.0,\n \"l2\" : 0.0,\n \"l2Bias\" : 0.0,\n \"layerName\" : \"out1\",\n \"learningRate\" : 0.1,\n \"learningRateSchedule\" : null,\n \"lossFn\" : {\n \"NegativeLogLikelihood\" : {\n \"softmaxClipEps\" : 1.0E-10\n }\n },\n \"lossFunction\" : \"NEGATIVELOGLIKELIHOOD\",\n \"momentum\" : 0.9,\n \"momentumSchedule\" : { },\n \"nin\" : 4,\n \"nout\" : 3,\n \"rho\" : \"NaN\",\n \"rmsDecay\" : \"NaN\",\n \"updater\" : \"NESTEROVS\",\n \"weightInit\" : \"XAVIER\"\n }\n },\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"variables\" : [ ]\n },\n \"outputVertex\" : false,\n \"preProcessor\" : null\n }\n },\n \"out2\" : {\n \"LayerVertex\" : {\n \"layerConf\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : {\n \"output\" : {\n \"activationFn\" : {\n \"Sigmoid\" : { }\n },\n \"adamMeanDecay\" : \"NaN\",\n \"adamVarDecay\" : \"NaN\",\n \"biasInit\" : 0.0,\n \"biasLearningRate\" : 0.1,\n \"dist\" : null,\n \"dropOut\" : 0.0,\n \"epsilon\" : \"NaN\",\n \"gradientNormalization\" : \"None\",\n \"gradientNormalizationThreshold\" : 1.0,\n \"iupdater\" : {\n \"@class\" : \"org.nd4j.linalg.learning.config.Nesterovs\",\n \"learningRate\" : 0.1,\n \"momentum\" : 0.9\n },\n \"l1\" : 0.0,\n \"l1Bias\" : 0.0,\n \"l2\" : 0.0,\n \"l2Bias\" : 0.0,\n \"layerName\" : \"out2\",\n \"learningRate\" : 0.1,\n \"learningRateSchedule\" : null,\n \"lossFn\" : {\n \"MSE\" : { }\n },\n \"lossFunction\" : \"MSE\",\n \"momentum\" : 0.9,\n \"momentumSchedule\" : { },\n \"nin\" : 4,\n \"nout\" : 2,\n \"rho\" : \"NaN\",\n \"rmsDecay\" : \"NaN\",\n \"updater\" : \"NESTEROVS\",\n \"weightInit\" : \"XAVIER\"\n }\n },\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 123,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"variables\" : [ ]\n },\n \"outputVertex\" : false,\n \"preProcessor\" : null\n }\n }\n }\n}\n"}]},"apps":[],"jobName":"paragraph_1508380714044_-127932388","id":"20171006-040530_601765710","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5331","user":"anonymous","dateFinished":"2017-10-19T02:47:27+0000","dateStarted":"2017-10-19T02:47:27+0000"},{"title":"","text":"%md\n### Creating a ComputationGraph\nFinally, to create a 'ComputationGraph', we pass the configuration to it as shown below","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Creating a ComputationGraph</h3>\n<p>Finally, to create a ‘ComputationGraph’, we pass the configuration to it as shown below</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714045_1151412711","id":"20171006-040532_1698179355","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5332","user":"anonymous","dateFinished":"2017-10-19T02:47:28+0000","dateStarted":"2017-10-19T02:47:28+0000"},{"title":"","text":"val computationGraph : ComputationGraph = new ComputationGraph(computationGraphConf)","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"computationGraph: org.deeplearning4j.nn.graph.ComputationGraph = org.deeplearning4j.nn.graph.ComputationGraph@721e59bd\n"}]},"apps":[],"jobName":"paragraph_1508380714049_-1882595306","id":"20171006-040534_1681194931","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5333","user":"anonymous","dateFinished":"2017-10-19T02:47:28+0000","dateStarted":"2017-10-19T02:47:28+0000"},{"text":"%md\n\n### More MultiLayerConfiguration Examples\n---\n\n### 1. Regularization","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>More MultiLayerConfiguration Examples</h3>\n<hr/>\n<h3>1. Regularization</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714049_-1447734467","id":"20171012-034725_1537519624","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5334","user":"anonymous","dateFinished":"2017-10-19T02:47:28+0000","dateStarted":"2017-10-19T02:47:28+0000"},{"title":"","text":"//You can add regularization in the higher level configuration in the network through first allowing regularization through 'regularization(true)' and then chaining it to a regularization algorithm -> 'l1()', l2()' etc as shown below:\r\nnew NeuralNetConfiguration.Builder().regularization(true).l2(1e-4)","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"res3: org.deeplearning4j.nn.conf.NeuralNetConfiguration.Builder = NeuralNetConfiguration.Builder(activationFn=sigmoid, weightInit=XAVIER, biasInit=0.0, dist=null, learningRate=0.1, biasLearningRate=NaN, learningRateSchedule=null, lrScoreBasedDecay=0.0, l1=NaN, l2=1.0E-4, l1Bias=NaN, l2Bias=NaN, dropOut=0.0, updater=SGD, iUpdater=Sgd(learningRate=0.001), momentum=NaN, momentumSchedule=null, epsilon=NaN, rho=NaN, rmsDecay=NaN, adamMeanDecay=NaN, adamVarDecay=NaN, layer=null, leakyreluAlpha=0.01, miniBatch=true, numIterations=1, maxNumLineSearchIterations=5, seed=1508381249178, useRegularization=true, optimizationAlgo=STOCHASTIC_GRADIENT_DESCENT, stepFunction=null, useDropConnect=false, minimize=true, gradientNormalization=None, gradientNormalizationThreshold=1.0, learningRatePolicy=None, ..."}]},"apps":[],"jobName":"paragraph_1508380714049_953586626","id":"20171005-003629_722905215","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5335","user":"anonymous","dateFinished":"2017-10-19T02:47:29+0000","dateStarted":"2017-10-19T02:47:28+0000"},{"text":"%md\n\n### 2. Dropout connects","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>2. Dropout connects</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714050_-1723932113","id":"20171012-034742_934801339","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5336","user":"anonymous","dateFinished":"2017-10-19T02:47:29+0000","dateStarted":"2017-10-19T02:47:29+0000"},{"title":"","text":"//When creating layers, you can add a dropout connection by using 'dropout(<dropOut_factor>)'\r\nnew NeuralNetConfiguration.Builder()\r\n .list() \r\n .layer(0, new DenseLayer.Builder().dropOut(0.8).build())","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"res5: org.deeplearning4j.nn.conf.NeuralNetConfiguration.ListBuilder = MultiLayerConfiguration.Builder(confs=[], dampingFactor=100.0, inputPreProcessors={}, pretrain=false, backprop=true, backpropType=Standard, tbpttFwdLength=20, tbpttBackLength=20, inputType=null, trainingWorkspaceMode=NONE, inferenceWorkspaceMode=SEPARATE, cacheMode=NONE)\n"}]},"apps":[],"jobName":"paragraph_1508380714050_285831816","id":"20171005-003645_1033200218","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5337","user":"anonymous","dateFinished":"2017-10-19T02:47:29+0000","dateStarted":"2017-10-19T02:47:29+0000"},{"text":"%md \n\n### 3. Bias initialization","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>3. Bias initialization</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714050_-1766411473","id":"20171012-034830_527315303","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5338","user":"anonymous","dateFinished":"2017-10-19T02:47:29+0000","dateStarted":"2017-10-19T02:47:29+0000"},{"title":"","text":"//You can initialize the bias of a particular layer by using 'biasInit(<init_value>)'\nnew NeuralNetConfiguration.Builder()\n .list() \n .layer(0, new DenseLayer.Builder().biasInit(0).build())","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"res7: org.deeplearning4j.nn.conf.NeuralNetConfiguration.ListBuilder = MultiLayerConfiguration.Builder(confs=[], dampingFactor=100.0, inputPreProcessors={}, pretrain=false, backprop=true, backpropType=Standard, tbpttFwdLength=20, tbpttBackLength=20, inputType=null, trainingWorkspaceMode=NONE, inferenceWorkspaceMode=SEPARATE, cacheMode=NONE)\n"}]},"apps":[],"jobName":"paragraph_1508380714051_-964860911","id":"20171005-003647_1870614088","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5339","user":"anonymous","dateFinished":"2017-10-19T02:47:30+0000","dateStarted":"2017-10-19T02:47:29+0000"},{"text":"%md\n\n### More ComputationGraphConfiguration Examples\n---\n\n### 1. Recurrent Network with Skip Connections","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>More ComputationGraphConfiguration Examples</h3>\n<hr/>\n<h3>1. Recurrent Network with Skip Connections</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714051_2013516981","id":"20171012-034538_1971807661","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5340","user":"anonymous","dateFinished":"2017-10-19T02:47:30+0000","dateStarted":"2017-10-19T02:47:30+0000"},{"title":"","text":"val cgConf1 : ComputationGraphConfiguration = new NeuralNetConfiguration.Builder()\r\n .learningRate(0.01)\r\n .graphBuilder()\r\n .addInputs(\"input\") //can use any label for this\r\n .addLayer(\"L1\", new GravesLSTM.Builder().nIn(5).nOut(5).build(), \"input\")\r\n .addLayer(\"L2\",new RnnOutputLayer.Builder().nIn(5+5).nOut(5).build(), \"input\", \"L1\")\r\n .setOutputs(\"L2\")\r\n .build();","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{"0":{"graph":{"mode":"table","height":334,"optionOpen":false}}},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"cgConf1: org.deeplearning4j.nn.conf.ComputationGraphConfiguration =\n{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"defaultConfiguration\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : null,\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 1508381250961,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"vari..."}]},"apps":[],"jobName":"paragraph_1508380714051_1064390857","id":"20171005-003820_61703775","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5341","user":"anonymous","dateFinished":"2017-10-19T02:47:31+0000","dateStarted":"2017-10-19T02:47:30+0000"},{"text":"%md\n### 2. Multiple Inputs and Merge Vertex","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>2. Multiple Inputs and Merge Vertex</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714052_-40620558","id":"20171012-034559_1684008513","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5342","user":"anonymous","dateFinished":"2017-10-19T02:47:30+0000","dateStarted":"2017-10-19T02:47:30+0000"},{"title":"","text":"//Here MergeVertex concatenates the layer outputs\r\nval cgConf2 : ComputationGraphConfiguration = new NeuralNetConfiguration.Builder()\r\n .learningRate(0.01)\r\n .graphBuilder()\r\n .addInputs(\"input1\", \"input2\")\r\n .addLayer(\"L1\", new DenseLayer.Builder().nIn(3).nOut(4).build(), \"input1\")\r\n .addLayer(\"L2\", new DenseLayer.Builder().nIn(3).nOut(4).build(), \"input2\")\r\n .addVertex(\"merge\", new MergeVertex(), \"L1\", \"L2\")\r\n .addLayer(\"out\", new OutputLayer.Builder().nIn(4+4).nOut(3).build(), \"merge\")\r\n .setOutputs(\"out\")\r\n .build();","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{"0":{"graph":{"mode":"table","height":354,"optionOpen":false}}},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"cgConf2: org.deeplearning4j.nn.conf.ComputationGraphConfiguration =\n{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"defaultConfiguration\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : null,\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 1508381251741,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"vari..."}]},"apps":[],"jobName":"paragraph_1508380714052_-1928728493","id":"20171005-003839_1750323004","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5343","user":"anonymous","dateFinished":"2017-10-19T02:47:31+0000","dateStarted":"2017-10-19T02:47:31+0000"},{"text":"%md\n\n### 3. Multi-Task Learning","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>3. Multi-Task Learning</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714052_-1730811254","id":"20171012-034629_1111232997","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5344","user":"anonymous","dateFinished":"2017-10-19T02:47:31+0000","dateStarted":"2017-10-19T02:47:31+0000"},{"title":"","text":"val cgConf3 : ComputationGraphConfiguration = new NeuralNetConfiguration.Builder()\r\n .learningRate(0.01)\r\n .graphBuilder()\r\n .addInputs(\"input\")\r\n .addLayer(\"L1\", new DenseLayer.Builder().nIn(3).nOut(4).build(), \"input\")\r\n .addLayer(\"out1\", new OutputLayer.Builder()\r\n .lossFunction(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)\r\n .nIn(4).nOut(3).build(), \"L1\")\r\n .addLayer(\"out2\", new OutputLayer.Builder()\r\n .lossFunction(LossFunctions.LossFunction.MSE)\r\n .nIn(4).nOut(2).build(), \"L1\")\r\n .setOutputs(\"out1\",\"out2\")\r\n .build();","dateUpdated":"2017-10-19T02:45:40+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"cgConf3: org.deeplearning4j.nn.conf.ComputationGraphConfiguration =\n{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"defaultConfiguration\" : {\n \"cacheMode\" : \"NONE\",\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : null,\n \"leakyreluAlpha\" : 0.0,\n \"learningRateByParam\" : { },\n \"learningRatePolicy\" : \"None\",\n \"lrPolicyDecayRate\" : \"NaN\",\n \"lrPolicyPower\" : \"NaN\",\n \"lrPolicySteps\" : \"NaN\",\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"numIterations\" : 1,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 1508381252343,\n \"stepFunction\" : null,\n \"useDropConnect\" : false,\n \"useRegularization\" : false,\n \"vari..."}]},"apps":[],"jobName":"paragraph_1508380714053_1846805427","id":"20171005-003841_25082385","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5345","user":"anonymous","dateFinished":"2017-10-19T02:47:32+0000","dateStarted":"2017-10-19T02:47:31+0000"},{"title":"","text":"%md\n\n### What's next?\n---\n\n- See tutorial [here](http://someurl) to learn about different ways to feed training data to a network\n- See tutorial [here](http://someurl) to learn about how to fit a network to a specified configuration and training data\n","dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"fontSize":9,"title":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>What’s next?</h3>\n<hr/>\n<ul>\n <li>See tutorial <a href=\"http://someurl\">here</a> to learn about different ways to feed training data to a network</li>\n <li>See tutorial <a href=\"http://someurl\">here</a> to learn about how to fit a network to a specified configuration and training data</li>\n</ul>\n</div>"}]},"apps":[],"jobName":"paragraph_1508380714053_-1339405078","id":"20171006-040048_813348401","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:5346","user":"anonymous","dateFinished":"2017-10-19T02:47:32+0000","dateStarted":"2017-10-19T02:47:31+0000"},{"dateUpdated":"2017-10-19T02:38:34+0000","config":{"tableHide":true,"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1508380714054_412115831","id":"20171008-172049_618031942","dateCreated":"2017-10-19T02:38:34+0000","status":"FINISHED","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:5347","user":"anonymous"}],"name":"MultiLayerNetwork and ComputationGraph","id":"2CY5BTUXG","angularObjects":{"2CYJ4NZN3:shared_process":[],"2CYMT2PG8:shared_process":[],"2CXZZB5VZ:shared_process":[],"2CW6ABJTT:shared_process":[],"2CUYEYH4S:shared_process":[],"2CY668KU3:shared_process":[],"2CWWMXHKF:shared_process":[],"2CUVQN82F:shared_process":[],"2CXBEEV32:shared_process":[],"2CYBUT9QX:shared_process":[],"2CX8Y7CXR:shared_process":[],"2CVE9XZTV:shared_process":[],"2CW88YH9F:shared_process":[],"2CXYHKN86:shared_process":[],"2CYKW1GFV:shared_process":[],"2CWPVCH52:shared_process":[],"2CWE9VKMB:shared_process":[],"2CWXNDB5J:shared_process":[],"2CWKUKFKX:shared_process":[],"2CVAHGVDC:shared_process":[]},"config":{"looknfeel":"default","personalizedMode":"false"},"info":{}}