{"paragraphs":[{"text":"%md\n### Note\n\nPlease view the [README](https://github.com/deeplearning4j/dl4j-examples/blob/master/tutorials/README.md) to learn about installing, setting up dependencies, and importing notebooks in Zeppelin","user":"anonymous","dateUpdated":"2017-11-11T21:58:15+0000","config":{"colWidth":12,"fontSize":9,"enabled":true,"results":{},"editorSetting":{"language":"markdown","editOnDblClick":true},"editorMode":"ace/mode/markdown","editorHide":true,"tableHide":false},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Note</h3>\n<p>Please view the <a href=\"https://github.com/deeplearning4j/dl4j-examples/blob/master/tutorials/README.md\">README</a> to learn about installing, setting up dependencies, and importing notebooks in Zeppelin</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437491543_-1275440392","id":"20171111-215811_1487615004","dateCreated":"2017-11-11T21:58:11+0000","dateStarted":"2017-11-11T21:58:15+0000","dateFinished":"2017-11-11T21:58:19+0000","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:1662"},{"text":"%md\n\n### Background\n\nIn previous tutorials we learned how to configure different neural networks such as feed forward, convolutional, and recurrent networks. The type of neural network is determined by the type of hidden layers they contain. For example, feed forward neural networks are comprised of dense layers, while recurrent neural networks can include Graves LSTM (long short-term memory) layers. In this tutorial we will learn how to use combinations of different layers in a single neural network using the MultiLayerNetwork class of deeplearning4j (DL4J). Additionally, we will learn how to use preprocess our data to more efficiently train the neural networks. The MNIST dataset (images of handwritten digits) will be used as an example for a convolutional network. ","dateUpdated":"2017-11-11T21:56:08+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Background</h3>\n<p>In previous tutorials we learned how to configure different neural networks such as feed forward, convolutional, and recurrent networks. The type of neural network is determined by the type of hidden layers they contain. For example, feed forward neural networks are comprised of dense layers, while recurrent neural networks can include Graves LSTM (long short-term memory) layers. In this tutorial we will learn how to use combinations of different layers in a single neural network using the MultiLayerNetwork class of deeplearning4j (DL4J). Additionally, we will learn how to use preprocess our data to more efficiently train the neural networks. The MNIST dataset (images of handwritten digits) will be used as an example for a convolutional network.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437368694_-10024342","id":"20171104-011438_1504233936","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1663"},{"text":"%md\n### Imports","dateUpdated":"2017-11-11T21:56:08+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Imports</h3>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437368706_1774473527","id":"20171104-011622_1428528099","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1664"},{"text":"\nimport org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator\nimport org.deeplearning4j.eval.Evaluation\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm\nimport org.deeplearning4j.nn.conf.MultiLayerConfiguration\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration\nimport org.deeplearning4j.nn.conf.Updater\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.deeplearning4j.nn.conf.layers.SubsamplingLayer\nimport org.deeplearning4j.nn.conf.layers.ConvolutionLayer\nimport org.deeplearning4j.nn.conf.inputs.InputType\nimport org.deeplearning4j.eval.Evaluation\nimport org.deeplearning4j.nn.conf.distribution.UniformDistribution\nimport org.deeplearning4j.nn.conf.layers.{DenseLayer, OutputLayer}\nimport org.deeplearning4j.nn.conf.{ComputationGraphConfiguration, MultiLayerConfiguration, NeuralNetConfiguration, Updater}\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.learning.config.Nesterovs\nimport org.nd4j.linalg.lossfunctions.LossFunctions\nimport org.nd4j.linalg.api.ndarray.INDArray\nimport org.nd4j.linalg.dataset.DataSet\nimport org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.dataset.api.iterator.DataSetIterator\nimport org.nd4j.linalg.dataset.api.preprocessor.DataNormalization\nimport org.nd4j.linalg.dataset.api.preprocessor.ImagePreProcessingScaler\nimport org.slf4j.Logger\nimport org.slf4j.LoggerFactory","dateUpdated":"2017-11-11T21:56:08+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"import org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator\nimport org.deeplearning4j.eval.Evaluation\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm\nimport org.deeplearning4j.nn.conf.MultiLayerConfiguration\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration\nimport org.deeplearning4j.nn.conf.Updater\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.deeplearning4j.nn.conf.layers.SubsamplingLayer\nimport org.deeplearning4j.nn.conf.layers.ConvolutionLayer\nimport org.deeplearning4j.nn.conf.inputs.InputType\nimport org.deeplearning4j.eval.Evaluation\nimport org.deeplearning4j.nn.conf.distribution.UniformDistribution\nimport org.deeplearning4j.nn.conf.layers.{DenseLayer, OutputLayer}\nimport org.deeplearning4j.nn.conf.{ComputationGraphConfiguration, MultiLayerConfiguration, NeuralNetConfiguration, Updater}\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.learning.config.Nesterovs\nimport org.nd4j.linalg.lossfunctions.LossFunctions\nimport org.nd4j.linalg.api.ndarray.INDArray\nimport org.nd4j.linalg.dataset.DataSet\nimport org.nd4j.linalg.lossfunctions.LossFunctions.LossFunction\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.dataset.api.iterator.DataSetIterator\nimport org.nd4j.linalg.dataset.api.preprocessor.DataNormalization\nimport org.nd4j.linalg.dataset.api.preprocessor.ImagePreProcessingScaler\nimport org.slf4j.Logger\nimport org.slf4j.LoggerFactory\n"}]},"apps":[],"jobName":"paragraph_1510437368706_771417731","id":"20171104-011824_1029004641","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1665"},{"text":"%md\n\n### Convolutional Neural Network Example\n\nNow that everything needed is imported, we can start by configuring a convolutional neural network for a MultiLayerNetwork. This network will consist of two convolutional layers, two max pooling layers, one dense layer, and an output layer. This is easy to do using DL4J's functionality; we simply add a dense layer after the max pooling layer to convert the output into vectorized form before passing it to the output layer. The neural network will then attempt to classify an observation using the vectorized data in the output layer. \n\nThe only tricky part is getting the dimensions of the input to the dense layer correctly after the convolutional and max pooling layers. Note that we first start off with a 28 by 28 matrix and after applying the convolution layer with a 5 by 5 kernel we end up with twenty 24 by 24 matrices. Once the input is passed through the max pooling layer with a 2 by 2 kernel and a stride of 2 by 2, we end up with twenty 12 by 12 matrices. After the second convolutional layer with a 5 by 5 kernel, we end up with fifty 8 by 8 matrices. This output is reduced to fifty 4 by 4 matrices after the second max pooling layer which has the same kernel size and stride of the first max pooling layer. To vectorize these final matrices, we require an input of dimension 50*4*4 or 800 in the dense layer.","user":"anonymous","dateUpdated":"2017-11-11T22:03:51+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<h3>Convolutional Neural Network Example</h3>\n<p>Now that everything needed is imported, we can start by configuring a convolutional neural network for a MultiLayerNetwork. This network will consist of two convolutional layers, two max pooling layers, one dense layer, and an output layer. This is easy to do using DL4J&rsquo;s functionality; we simply add a dense layer after the max pooling layer to convert the output into vectorized form before passing it to the output layer. The neural network will then attempt to classify an observation using the vectorized data in the output layer. </p>\n<p>The only tricky part is getting the dimensions of the input to the dense layer correctly after the convolutional and max pooling layers. Note that we first start off with a 28 by 28 matrix and after applying the convolution layer with a 5 by 5 kernel we end up with twenty 24 by 24 matrices. Once the input is passed through the max pooling layer with a 2 by 2 kernel and a stride of 2 by 2, we end up with twenty 12 by 12 matrices. After the second convolutional layer with a 5 by 5 kernel, we end up with fifty 8 by 8 matrices. This output is reduced to fifty 4 by 4 matrices after the second max pooling layer which has the same kernel size and stride of the first max pooling layer. To vectorize these final matrices, we require an input of dimension 50*4*4 or 800 in the dense layer.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437368707_634544981","id":"20171104-013013_450370534","dateCreated":"2017-11-11T21:56:08+0000","dateStarted":"2017-11-11T22:03:51+0000","dateFinished":"2017-11-11T22:03:51+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:1666"},{"text":"val nChannels = 1; // Number of input channels\nval outputNum = 10; // The number of possible outcomes\nval batchSize = 64; // Test batch size\nval nEpochs = 1; // Number of training epochs\nval iterations = 1; // Number of training iterations\nval seed = 123; // Random seed\n\nval conf : MultiLayerConfiguration = new NeuralNetConfiguration.Builder()\n    .seed(12345)\n    .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n    .list()\n    .layer(0, new ConvolutionLayer.Builder(5, 5)\n        .nIn(1)\n        .stride(1, 1)\n        .nOut(20)\n        .activation(Activation.IDENTITY)\n        .build())\n    .layer(1, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)\n        .kernelSize(2,2)\n        .stride(2,2)\n        .build())\n     .layer(2, new ConvolutionLayer.Builder(5, 5)\n        .stride(1, 1)\n        .nOut(50)\n        .activation(Activation.IDENTITY)\n        .build())\n    .layer(3, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)\n        .kernelSize(2,2)\n        .stride(2,2)\n        .build())\n    .layer(4, new DenseLayer.Builder().activation(Activation.RELU)\n        .nIn(800)\n        .nOut(500).build())\n    .layer(5, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)\n        .nIn(500)\n        .nOut(outputNum)\n        .activation(Activation.SOFTMAX)\n        .build())\n    .setInputType(InputType.convolutionalFlat(28,28,1)) \n\t.backprop(true).pretrain(false).build()\n\t\nval model = new MultiLayerNetwork(conf)","dateUpdated":"2017-11-11T21:56:08+0000","config":{"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"nChannels: Int = 1\noutputNum: Int = 10\nbatchSize: Int = 64\nnEpochs: Int = 1\niterations: Int = 1\nseed: Int = 123\nconf: org.deeplearning4j.nn.conf.MultiLayerConfiguration =\n{\n  \"backprop\" : true,\n  \"backpropType\" : \"Standard\",\n  \"cacheMode\" : \"NONE\",\n  \"confs\" : [ {\n    \"cacheMode\" : \"NONE\",\n    \"iterationCount\" : 0,\n    \"l1ByParam\" : { },\n    \"l2ByParam\" : { },\n    \"layer\" : {\n      \"convolution\" : {\n        \"activationFn\" : {\n          \"Identity\" : { }\n        },\n        \"adamMeanDecay\" : \"NaN\",\n        \"adamVarDecay\" : \"NaN\",\n        \"biasInit\" : 0.0,\n        \"biasLearningRate\" : 0.1,\n        \"convolutionMode\" : \"Truncate\",\n        \"cudnnAlgoMode\" : \"PREFER_FASTEST\",\n        \"cudnnBwdDataAlgo\" : null,\n        \"cudnnBwdFilterAlgo\" : null,\n        \"cudnnFwdAlgo\" : null,\n        \"dist\" : null,\n        \"dropOut\" : 0.0,\n        \"epsilon\" : \"NaN\",\n        \"gradientNormalization\" : \"None\",\n        \"gra...model: org.deeplearning4j.nn.multilayer.MultiLayerNetwork = org.deeplearning4j.nn.multilayer.MultiLayerNetwork@6c296f44\n"}]},"apps":[],"jobName":"paragraph_1510437368707_1464202944","id":"20171104-011836_485502725","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1667"},{"text":"%md\nBefore training the neural network, we will instantiate built-in DataSetIterators for the MNIST data. One example of data preprocessing is scaling the data. The data we are using in raw form are greyscale images, which are represented by a single matrix filled with integer values from 0 to 255. A 0 value indicates a black pixel, while a 1 value indicates a white pixel. It is helpful to scale the image pixel value from 0 to 1 instead of from 0 to 255. To do this, the ImagePreProcessingScaler class is used directly on the MnistDataSetIterators. Note that this process is typtical for data preprocessing. Once this is done, we are ready to train the neural network.","dateUpdated":"2017-11-11T22:04:39+0000","config":{"tableHide":true,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Before training the neural network, we will instantiate built-in DataSetIterators for the MNIST data. One example of data preprocessing is scaling the data. The data we are using in raw form are greyscale images, which are represented by a single matrix filled with integer values from 0 to 255. A 0 value indicates a black pixel, while a 1 value indicates a white pixel. It is helpful to scale the image pixel value from 0 to 1 instead of from 0 to 255. To do this, the ImagePreProcessingScaler class is used directly on the MnistDataSetIterators. Once this is done, we are ready to train the neural network.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437368708_605934324","id":"20171104-012137_1301512944","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1668"},{"text":"val rngSeed = 12345\nval mnistTrain = new MnistDataSetIterator(batchSize, true, rngSeed)\nval mnistTest = new MnistDataSetIterator(batchSize, false, rngSeed)\n\nval scaler : DataNormalization = new ImagePreProcessingScaler(0,1);\nscaler.fit(mnistTrain);\nmnistTrain.setPreProcessor(scaler);\nmnistTest.setPreProcessor(scaler);\n\n","dateUpdated":"2017-11-11T21:56:08+0000","config":{"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"rngSeed: Int = 12345\nmnistTrain: org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator = org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator@1aef8e39\nmnistTest: org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator = org.deeplearning4j.datasets.iterator.impl.MnistDataSetIterator@103f2f0c\nscaler: org.nd4j.linalg.dataset.api.preprocessor.DataNormalization = org.nd4j.linalg.dataset.api.preprocessor.ImagePreProcessingScaler@168dbfa3\n"}]},"apps":[],"jobName":"paragraph_1510437368708_-1108447068","id":"20171104-012813_280696259","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1669"},{"text":"%md\nTo train the neural network, we use 5 epochs or complete passes through the training set by simply calling the fit method.","dateUpdated":"2017-11-11T21:56:08+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>To train the neural network, we use 5 epochs or complete passes through the training set by simply calling the fit method.</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437368709_-1724559736","id":"20171105-081158_1002279647","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1670"},{"text":"val nEpochs = 5\n\n(1 to nEpochs).foreach{ epoch =>\n    model.fit(mnistTrain)\n    println(\"Epoch \" + epoch + \" complete\")\n}\n","dateUpdated":"2017-11-11T21:56:08+0000","config":{"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"nEpochs: Int = 5\nEpoch 1 complete\nEpoch 2 complete\nEpoch 3 complete\nEpoch 4 complete\nEpoch 5 complete\n"}]},"apps":[],"jobName":"paragraph_1510437368709_478347810","id":"20171105-081156_507863200","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1671"},{"text":"%md\n\nLastly, we use the test split of the data to evaluate how well our final model performs on data it has never seen. We can see that the model performs pretty well using only 5 epochs!","user":"anonymous","dateUpdated":"2017-11-11T22:01:57+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<div class=\"markdown-body\">\n<p>Lastly, we use the test split of the data to evaluate how well our final model performs on data it has never seen. We can see that the model performs pretty well using only 5 epochs!</p>\n</div>"}]},"apps":[],"jobName":"paragraph_1510437368710_78131255","id":"20171105-080801_188004903","dateCreated":"2017-11-11T21:56:08+0000","dateStarted":"2017-11-11T22:01:57+0000","dateFinished":"2017-11-11T22:01:57+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:1672"},{"text":"val eval : Evaluation = model.evaluate(mnistTest)\nprintln(eval.stats())","dateUpdated":"2017-11-11T21:56:08+0000","config":{"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"eval: org.deeplearning4j.eval.Evaluation =\n\nExamples labeled as 0 classified by model as 0: 942 times\nExamples labeled as 0 classified by model as 5: 15 times\nExamples labeled as 0 classified by model as 6: 2 times\nExamples labeled as 0 classified by model as 7: 1 times\nExamples labeled as 0 classified by model as 8: 19 times\nExamples labeled as 0 classified by model as 9: 1 times\nExamples labeled as 1 classified by model as 1: 972 times\nExamples labeled as 1 classified by model as 2: 3 times\nExamples labeled as 1 classified by model as 3: 11 times\nExamples labeled as 1 classified by model as 5: 8 times\nExamples labeled as 1 classified by model as 6: 3 times\nExamples labeled as 1 classified by model as 8: 138 times\nExamples labeled as 2 classified by model as 0: 25 times\nExamples labele...\nExamples labeled as 0 classified by model as 0: 942 times\nExamples labeled as 0 classified by model as 5: 15 times\nExamples labeled as 0 classified by model as 6: 2 times\nExamples labeled as 0 classified by model as 7: 1 times\nExamples labeled as 0 classified by model as 8: 19 times\nExamples labeled as 0 classified by model as 9: 1 times\nExamples labeled as 1 classified by model as 1: 972 times\nExamples labeled as 1 classified by model as 2: 3 times\nExamples labeled as 1 classified by model as 3: 11 times\nExamples labeled as 1 classified by model as 5: 8 times\nExamples labeled as 1 classified by model as 6: 3 times\nExamples labeled as 1 classified by model as 8: 138 times\nExamples labeled as 2 classified by model as 0: 25 times\nExamples labeled as 2 classified by model as 1: 2 times\nExamples labeled as 2 classified by model as 2: 841 times\nExamples labeled as 2 classified by model as 3: 7 times\nExamples labeled as 2 classified by model as 4: 21 times\nExamples labeled as 2 classified by model as 5: 3 times\nExamples labeled as 2 classified by model as 6: 30 times\nExamples labeled as 2 classified by model as 7: 10 times\nExamples labeled as 2 classified by model as 8: 89 times\nExamples labeled as 2 classified by model as 9: 4 times\nExamples labeled as 3 classified by model as 0: 5 times\nExamples labeled as 3 classified by model as 2: 17 times\nExamples labeled as 3 classified by model as 3: 799 times\nExamples labeled as 3 classified by model as 4: 1 times\nExamples labeled as 3 classified by model as 5: 68 times\nExamples labeled as 3 classified by model as 6: 1 times\nExamples labeled as 3 classified by model as 7: 21 times\nExamples labeled as 3 classified by model as 8: 92 times\nExamples labeled as 3 classified by model as 9: 6 times\nExamples labeled as 4 classified by model as 0: 1 times\nExamples labeled as 4 classified by model as 2: 2 times\nExamples labeled as 4 classified by model as 3: 1 times\nExamples labeled as 4 classified by model as 4: 779 times\nExamples labeled as 4 classified by model as 6: 35 times\nExamples labeled as 4 classified by model as 8: 52 times\nExamples labeled as 4 classified by model as 9: 112 times\nExamples labeled as 5 classified by model as 0: 10 times\nExamples labeled as 5 classified by model as 1: 1 times\nExamples labeled as 5 classified by model as 2: 4 times\nExamples labeled as 5 classified by model as 3: 32 times\nExamples labeled as 5 classified by model as 4: 7 times\nExamples labeled as 5 classified by model as 5: 751 times\nExamples labeled as 5 classified by model as 6: 12 times\nExamples labeled as 5 classified by model as 7: 2 times\nExamples labeled as 5 classified by model as 8: 69 times\nExamples labeled as 5 classified by model as 9: 4 times\nExamples labeled as 6 classified by model as 0: 17 times\nExamples labeled as 6 classified by model as 1: 1 times\nExamples labeled as 6 classified by model as 2: 9 times\nExamples labeled as 6 classified by model as 3: 1 times\nExamples labeled as 6 classified by model as 4: 23 times\nExamples labeled as 6 classified by model as 5: 47 times\nExamples labeled as 6 classified by model as 6: 827 times\nExamples labeled as 6 classified by model as 7: 1 times\nExamples labeled as 6 classified by model as 8: 31 times\nExamples labeled as 6 classified by model as 9: 1 times\nExamples labeled as 7 classified by model as 0: 2 times\nExamples labeled as 7 classified by model as 1: 5 times\nExamples labeled as 7 classified by model as 2: 29 times\nExamples labeled as 7 classified by model as 3: 7 times\nExamples labeled as 7 classified by model as 4: 4 times\nExamples labeled as 7 classified by model as 5: 10 times\nExamples labeled as 7 classified by model as 7: 868 times\nExamples labeled as 7 classified by model as 8: 55 times\nExamples labeled as 7 classified by model as 9: 48 times\nExamples labeled as 8 classified by model as 0: 7 times\nExamples labeled as 8 classified by model as 3: 10 times\nExamples labeled as 8 classified by model as 4: 10 times\nExamples labeled as 8 classified by model as 5: 29 times\nExamples labeled as 8 classified by model as 6: 5 times\nExamples labeled as 8 classified by model as 7: 9 times\nExamples labeled as 8 classified by model as 8: 898 times\nExamples labeled as 8 classified by model as 9: 6 times\nExamples labeled as 9 classified by model as 0: 10 times\nExamples labeled as 9 classified by model as 2: 3 times\nExamples labeled as 9 classified by model as 3: 7 times\nExamples labeled as 9 classified by model as 4: 51 times\nExamples labeled as 9 classified by model as 5: 17 times\nExamples labeled as 9 classified by model as 6: 2 times\nExamples labeled as 9 classified by model as 7: 24 times\nExamples labeled as 9 classified by model as 8: 108 times\nExamples labeled as 9 classified by model as 9: 787 times\n\n\n==========================Scores========================================\n # of classes:    10\n Accuracy:        0.8464\n Precision:       0.8637\n Recall:          0.8468\n F1 Score:        0.8495\nPrecision, recall & F1: macro-averaged (equally weighted avg. of 10 classes)\n========================================================================\n"}]},"apps":[],"jobName":"paragraph_1510437368710_286733017","id":"20171104-012959_58417290","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1673"},{"dateUpdated":"2017-11-11T21:56:08+0000","config":{"editorSetting":{"language":"scala"},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1510437368711_-1964085385","id":"20171108-040047_425972641","dateCreated":"2017-11-11T21:56:08+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:1674"}],"name":"Layers and Preprocessors","id":"2CYVSUW5X","angularObjects":{"2CXKS4RR8:shared_process":[],"2CXVH1GF7:shared_process":[],"2CZAEBQP7:shared_process":[],"2CX5WZYD4:shared_process":[],"2CXJPFVHW:shared_process":[],"2CZASDS5V:shared_process":[],"2D13EC3VH:shared_process":[],"2CXVEUZ3C:shared_process":[],"2CXATMVZK:shared_process":[],"2D11MFNK1:shared_process":[],"2CXWYW852:shared_process":[],"2CZMRBKWE:shared_process":[],"2CXY4JFHM:shared_process":[],"2D15TEF35:shared_process":[],"2CX3FQW2D:shared_process":[],"2CY988JQU:shared_process":[],"2CWSQUZRR:shared_process":[],"2CXYHPHQV:shared_process":[],"2CYC5QFNN:shared_process":[],"2CZ1CZ3TN:shared_process":[]},"config":{"looknfeel":"default","personalizedMode":"false"},"info":{}}