{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Note\n",
"\n",
"View the [README.md](https://github.com/deeplearning4j/dl4j-examples/blob/master/tutorials/README.md) to learn about installing, setting up dependencies and importing notebooks in Zeppelin."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Background\n",
"---\n",
"\n",
"With deep learning, we can compose a deep neural network to suit the input data and its features. The goal is to train the network on the data to make predictions, and those predictions are tied to the outcomes that you care about; i.e. is this transaction fraudulent or not, or which object is contained in the photo? There are different techniques to configure a neural network, and all of them build a relational hierarchy between the inputs and outputs.\n",
"\n",
"In this tutorial, we are going to configure the simplest neural network and that is logistic regression model network. \n",
"\n",
"Regression is a process that helps show the relations between the independant variables (inputs) and the dependant variables (outputs). Logistic regression is one in which the dependant variable is categorical rather than continuous - meaning that it can predict only a limited number of classes or categories, like a switch you flip on or off. For example, it can predict that an image contains a cat or a dog, or it can classify input in ten buckets with the integers 0 through 9.\n",
"\n",
"A simple logisitic regression calculates 'x*w + b = y'. Where 'x' is an isntance of input data, 'w' is the weight or coefficient that transforms that input, 'b' is the bias and 'y' is the output, or prediction about the data. The biological terms show how this artificial neuron loosely maps to a neuron in the human brain. The most important point is how data flows through and is transformed by this structure.\n",
"\n",
"|---|---|---|\n",
"|**Logistic Regression** |  | [Source](https://i.pinimg.com/736x/61/fe/81/61fe81589ab491d1d3ba612b3bdf5b51--convolutional-neural-network-neuron-model.jpg) |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What will we learn in this tutorial?\n",
"We're going to configure the simplest network, with just one input layer and one output layer, to show how logistic regression works."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Imports"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"autoscroll": "auto"
},
"outputs": [],
"source": [
"import org.deeplearning4j.nn.api.OptimizationAlgorithm\n",
"import org.deeplearning4j.nn.conf.graph.MergeVertex\n",
"import org.deeplearning4j.nn.conf.layers.{DenseLayer, GravesLSTM, OutputLayer, RnnOutputLayer}\n",
"import org.deeplearning4j.nn.conf.{ComputationGraphConfiguration, MultiLayerConfiguration, NeuralNetConfiguration}\n",
"import org.deeplearning4j.nn.graph.ComputationGraph\n",
"import org.deeplearning4j.nn.multilayer.MultiLayerNetwork\n",
"import org.deeplearning4j.nn.weights.WeightInit\n",
"import org.nd4j.linalg.activations.Activation\n",
"import org.nd4j.linalg.learning.config.Nesterovs\n",
"import org.nd4j.linalg.lossfunctions.LossFunctions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configuring logistic regression layers\n",
"We are going to first build the layers and then feed these layers into the network configuration."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"autoscroll": "auto"
},
"outputs": [],
"source": [
"//Building the output layer\n",
"val outputLayer : OutputLayer = new OutputLayer.Builder()\n",
" .nIn(784) //The number of inputs feed from the input layer\n",
" .nOut(10) //The number of output values the output layer is supposed to take\n",
" .weightInit(WeightInit.XAVIER) //The algorithm to use for weights initialization\n",
" .activation(Activation.SOFTMAX) //Softmax activate converts the output layer into a probability distribution\n",
" .build() //Building our output layer"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"autoscroll": "auto"
},
"outputs": [],
"source": [
"//Since this is a simple network with a stack of layers we're going to configure a MultiLayerNetwork\n",
"val logisticRegressionConf : MultiLayerConfiguration = new NeuralNetConfiguration.Builder()\n",
" .seed(123).learningRate(0.1).iterations(1).optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT).updater(new Nesterovs(0.9)) //High Level Configuration\n",
" .list() //For configuring MultiLayerNetwork we call the list method\n",
" .layer(0, outputLayer) // <----- output layer fed here\n",
" .pretrain(false).backprop(true) //Pretraining and Backprop Configuration\n",
" .build() //Building Configuration"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**This is how our configuration here looks like:**\n",
"\n",
"|---|---|---|\n",
"|**Logistic Regression** |  | [Source](https://isaacchanghau.github.io/images/deeplearning/activationfunction/softmax.png) |\n",
"---\n",
"The layer with x1, x2, x3, ..., xn is out input layer. While the one with z1, z2, z3, ..., zk is our output layer. See how the weights and biases are connected, and how softmax is applied to give the probability distribution.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Why we didn't build an input layer\n",
"---\n",
"You may be wondering why didn't we write any code for building our input layer. The input layer is only a set of inputs values fed into the network. It doesn't perform a calculation. It's just an input sequence (raw or pre-processed data) coming into the network, data to be trained on or to be evaluated. Later, we are going to work with data iterators, which feed input to a network in a specific pattern, and which can be thought of as an input layer of the network."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" \n",
"\n",
"### What's next?\n",
"- See this [tutorial](http://someurl) to learn about configuring a more complex network: a 'feedforward neural network'. We will also introduce the concept of hidden layers."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Spark 2.0.0 - Scala 2.11",
"language": "scala",
"name": "spark2-scala"
},
"language_info": {
"codemirror_mode": "text/x-scala",
"file_extension": ".scala",
"mimetype": "text/x-scala",
"name": "scala",
"pygments_lexer": "scala",
"version": "2.11.8"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
