Revision 1db48f3a735eb0fba06a7d503f080a7ead512604 authored by Artem Artemev on 11 July 2018, 12:50:44 UTC, committed by GitHub on 11 July 2018, 12:50:44 UTC
1 parent 707b195
models.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Handling models in GPflow\n",
"--\n",
"\n",
"*James Hensman November 2015, January 2016*,\n",
"*Artem Artemev December 2017*\n",
"\n",
"One of the key ingredients in GPflow is the model class, which allows the user to carefully control parameters. This notebook shows how some of these parameter control features work, and how to build your own model with GPflow. First we'll look at\n",
"\n",
" - How to view models and parameters\n",
" - How to set parameter values\n",
" - How to constrain parameters (e.g. variance > 0)\n",
" - How to fix model parameters\n",
" - How to apply priors to parameters\n",
" - How to optimize models\n",
"\n",
"Then we'll show how to build a simple logistic regression model, demonstrating the ease of the parameter framework. \n",
"\n",
"GPy users should feel right at home, but there are some small differences.\n",
"\n",
"First, let's deal with the usual notebook boilerplate and make a simple GP regression model. See the Regression notebook for specifics of the model: we just want some parameters to play with."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"import gpflow\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a very simple GPR model without building it in TensorFlow graph."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"\n",
"with gpflow.defer_build():\n",
" X = np.random.rand(20, 1)\n",
" Y = np.sin(12 * X) + 0.66 * np.cos(25 * X) + np.random.randn(20,1) * 0.01\n",
" m = gpflow.models.GPR(X, Y, kern=gpflow.kernels.Matern32(1) + gpflow.kernels.Linear(1))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Viewing, getting and setting parameters\n",
"You can display the state of the model in a terminal with `print m` (or `print(m)`), and by simply returning it in a notebook"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>GPR/kern/matern32/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/matern32/lengthscales</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/linear/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/likelihood/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape \\\n",
"GPR/kern/matern32/variance Parameter None +ve True () \n",
"GPR/kern/matern32/lengthscales Parameter None +ve True () \n",
"GPR/kern/linear/variance Parameter None +ve True () \n",
"GPR/likelihood/variance Parameter None +ve True () \n",
"\n",
" fixed_shape value \n",
"GPR/kern/matern32/variance True 1.0 \n",
"GPR/kern/matern32/lengthscales True 1.0 \n",
"GPR/kern/linear/variance True 1.0 \n",
"GPR/likelihood/variance True 1.0 "
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.as_pandas_table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This model has four parameters. The kernel is made of the sum of two parts: the RBF kernel has a variance parameter and a lengthscale parameter, the linear kernel only has a variance parameter. There is also a parameter controlling the variance of the noise, as part of the likelihood. \n",
"\n",
"All of the model variables have been initialized at one. Individual parameters can be accessed in the same way as they are displayed in the table: to see all the parameters that are part of the likelihood, do"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>GPR/likelihood/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape \\\n",
"GPR/likelihood/variance Parameter None +ve True () \n",
"\n",
" fixed_shape value \n",
"GPR/likelihood/variance True 1.0 "
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.likelihood.as_pandas_table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This gets more useful with more complex models!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To set the value of a parameter, just assign."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>GPR/kern/matern32/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/matern32/lengthscales</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/linear/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/likelihood/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.01</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape \\\n",
"GPR/kern/matern32/variance Parameter None +ve True () \n",
"GPR/kern/matern32/lengthscales Parameter None +ve True () \n",
"GPR/kern/linear/variance Parameter None +ve True () \n",
"GPR/likelihood/variance Parameter None +ve True () \n",
"\n",
" fixed_shape value \n",
"GPR/kern/matern32/variance True 1.0 \n",
"GPR/kern/matern32/lengthscales True 0.5 \n",
"GPR/kern/linear/variance True 1.0 \n",
"GPR/likelihood/variance True 0.01 "
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.kern.kernels[0].lengthscales = 0.5\n",
"m.likelihood.variance = 0.01\n",
"m.as_pandas_table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Constraints and trainable variables\n",
"\n",
"GPflow helpfully creates an unconstrained representation of all the variables. Above, all the variables are constrained positive (see right hand table column), the unconstrained representation is given by $\\alpha = \\log(\\exp(\\theta)-1)$."
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'GPR/kern/linear/variance': array(1.0),\n",
" 'GPR/kern/matern32/lengthscales': array(0.5),\n",
" 'GPR/kern/matern32/variance': array(1.0),\n",
" 'GPR/likelihood/variance': array(0.01)}"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.read_trainables()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Constraints are handled by the `Transform` classes. You might prefer the constrain $\\alpha = \\log(\\theta)$: this is easily done by changing setting the transform attribute on a parameter, with one simple condition - the model has not been compiled yet:"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'GPR/kern/linear/variance': array(1.0),\n",
" 'GPR/kern/matern32/lengthscales': array(0.5),\n",
" 'GPR/kern/matern32/variance': array(1.0),\n",
" 'GPR/likelihood/variance': array(0.01)}"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.kern.kernels[0].lengthscales.transform = gpflow.transforms.Exp()\n",
"m.read_trainables()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The second free parameter, representing unconstrained lengthscale has changed (though the lengthscale itself remains the same). Another helpful feature is the ability to fix parameters. This is done by simply setting the fixed boolean to True: a 'fixed' notice appears in the representation and the corresponding variable is removed from the free state."
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>GPR/kern/matern32/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/matern32/lengthscales</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>Exp</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/linear/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>False</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/likelihood/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.01</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape \\\n",
"GPR/kern/matern32/variance Parameter None +ve True () \n",
"GPR/kern/matern32/lengthscales Parameter None Exp True () \n",
"GPR/kern/linear/variance Parameter None +ve False () \n",
"GPR/likelihood/variance Parameter None +ve True () \n",
"\n",
" fixed_shape value \n",
"GPR/kern/matern32/variance True 1.0 \n",
"GPR/kern/matern32/lengthscales True 0.5 \n",
"GPR/kern/linear/variance True 1.0 \n",
"GPR/likelihood/variance True 0.01 "
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.kern.kernels[1].variance.trainable = False\n",
"m.as_pandas_table()"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'GPR/kern/matern32/lengthscales': array(0.5),\n",
" 'GPR/kern/matern32/variance': array(1.0),\n",
" 'GPR/likelihood/variance': array(0.01)}"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.read_trainables()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To unfix a parameter, just flip the boolean back. The transformation (+ve) reappears."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>GPR/kern/matern32/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/matern32/lengthscales</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>Exp</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/linear/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/likelihood/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.01</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape \\\n",
"GPR/kern/matern32/variance Parameter None +ve True () \n",
"GPR/kern/matern32/lengthscales Parameter None Exp True () \n",
"GPR/kern/linear/variance Parameter None +ve True () \n",
"GPR/likelihood/variance Parameter None +ve True () \n",
"\n",
" fixed_shape value \n",
"GPR/kern/matern32/variance True 1.0 \n",
"GPR/kern/matern32/lengthscales True 0.5 \n",
"GPR/kern/linear/variance True 1.0 \n",
"GPR/likelihood/variance True 0.01 "
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.kern.kernels[1].variance.trainable = True\n",
"m.as_pandas_table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Priors\n",
"\n",
"Priors are set just like transforms and fixes, using members of the `gpflow.priors.` module. Let's set a Gamma prior on the RBF-variance."
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>GPR/kern/matern32/variance</th>\n",
" <td>Parameter</td>\n",
" <td>Ga([ 2.],[ 3.])</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/matern32/lengthscales</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>Exp</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/kern/linear/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>1.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>GPR/likelihood/variance</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>+ve</td>\n",
" <td>True</td>\n",
" <td>()</td>\n",
" <td>True</td>\n",
" <td>0.01</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform \\\n",
"GPR/kern/matern32/variance Parameter Ga([ 2.],[ 3.]) +ve \n",
"GPR/kern/matern32/lengthscales Parameter None Exp \n",
"GPR/kern/linear/variance Parameter None +ve \n",
"GPR/likelihood/variance Parameter None +ve \n",
"\n",
" trainable shape fixed_shape value \n",
"GPR/kern/matern32/variance True () True 1.0 \n",
"GPR/kern/matern32/lengthscales True () True 0.5 \n",
"GPR/kern/linear/variance True () True 1.0 \n",
"GPR/likelihood/variance True () True 0.01 "
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m.kern.kernels[0].variance.prior = gpflow.priors.Gamma(2,3)\n",
"m.as_pandas_table()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Optimization\n",
"\n",
"Optimization is done by creating an instance of optimizer, in our case it is `gpflow.train.ScipyOptimizer`, which has optional arguments that are passed through to `scipy.optimize.minimize` (we minimize the negative log-likelihood) and calling `minimize` method of that optimizer with model as optimization target. Variables that have priors are MAP-estimated, others are ML, i.e. we add the log prior to the log likelihood."
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Optimization terminated with:\n",
" Message: b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'\n",
" Objective function value: 2.634113\n",
" Number of iterations: 34\n",
" Number of functions evaluations: 39\n"
]
}
],
"source": [
"m.compile()\n",
"opt = gpflow.train.ScipyOptimizer()\n",
"opt.minimize(m)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Building new models\n",
"\n",
"To build new models, you'll need to inherrit from `gpflow.models.Model`. Parameters are instantiated with `gpflow.Param`. You may also be interested in `gpflow.params.Parameterized` which acts as a 'container' of `Param`s (e.g. kernels are Parameterized). \n",
"\n",
"In this very simple demo, we'll implement linear multiclass classification. There will be two parameters: a weight matrix and a 'bias' (offset). The key thing to implement is the private `_build_likelihood` method, which should return a tensorflow scalar representing the (log) likelihood. Param objects can be used inside `_build_likelihood`: they will appear as appropriate (unconstrained) tensors. "
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"class LinearMulticlass(gpflow.models.Model):\n",
" def __init__(self, X, Y, name=None):\n",
" super().__init__(name=name) # always call the parent constructor\n",
" \n",
" self.X = X.copy() # X is a numpy array of inputs\n",
" self.Y = Y.copy() # Y is a 1-of-k representation of the labels\n",
" \n",
" self.num_data, self.input_dim = X.shape\n",
" _, self.num_classes = Y.shape\n",
" \n",
" #make some parameters\n",
" self.W = gpflow.Param(np.random.randn(self.input_dim, self.num_classes))\n",
" self.b = gpflow.Param(np.random.randn(self.num_classes))\n",
" \n",
" # ^^ You must make the parameters attributes of the class for\n",
" # them to be picked up by the model. i.e. this won't work:\n",
" #\n",
" # W = gpflow.Param(... <-- must be self.W\n",
" \n",
" @gpflow.params_as_tensors\n",
" def _build_likelihood(self): # takes no arguments\n",
" p = tf.nn.softmax(tf.matmul(self.X, self.W) + self.b) # Param variables are used as tensorflow arrays. \n",
" return tf.reduce_sum(tf.log(p) * self.Y) # be sure to return a scalar"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"...and that's it. Let's build a really simple demo to show that it works."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.PathCollection at 0x120f67a58>"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAFpCAYAAAB0/VUQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYFNW9//FPdfeszAwwNNuwKygqIuCCiBFw17gmpjRe\nTYz5hWhiNPfGeK/xZrveJMYsxkSzGGM0V6M5cV+IKC4ocUdFQZRdhGEbYGD2rc/vjwEzyCxdTHVX\nd9f79Tw+Ml2Hqu/zpWfm01WnTjnWWgEAAABhFgm6AAAAACBohGIAAACEHqEYAAAAoUcoBgAAQOgR\nigEAABB6hGIAAACEHqEYAAAAoUcoBgAAQOgRigEAABB6hGIAAACEXiyg4/JsaQAAAKSL09OAoEKx\nKisrfdlPPB5XVVWVL/sKA/rlDf3yhn55Q7+8oV/e0C9v6Jc32dSvioqKpMYxfQIAAAChRygGAABA\n6BGKAQAAEHqEYgAAAIQeoRgAAAChRygGAABA6BGKAQAAEHqBrVMMAAAyR0NNoxbc/5peuO8VbVq9\nRdFYVPtNHqXjL56uySdNUCTKeTTkNkIxAAAht37ZRv38ot9pW2X1Hq8vnv++Fs9/Xwcfe4Cu/OOX\nVVRaGFCFQOrxsQ8AgBDbWVWjGz//270CcUfvLVimWy+/U9baNFYGpBehGACAEHv6zy+oetOOHse9\n+/xSffDKyjRUBASDUAwAQEgl2hKa/9eXkx7/7N3/TGE1QLAIxQAAhNSOLTXasaUm6fEfvbc+hdUA\nwSIUAwAQUl7nCDOlGLmMUAwAQEj1HViqkv59kh4/7IAhKawGCBahGACAkIrGojrugqOTHj/romNS\nWA0QLEIxAAAhdtKlx6l0QEmP4w6cur8OPvaANFQEBINQDABAiJUP7aer775MZfGug/HYw0frytu/\nrEiE2IDcxRPtAAAIudGHjtCPn71W8//6subf+4o2f1ilSDSi/SaN1PEXH6upZ01WLJ/IgNzGOxwA\nAKi0vERnXHGSzrjiJCXaEnIijhzHCbosIG0IxQAAYA+RKNMkED686wEAABB6hGIAAACEHqEYAAAA\noUcoBgAAQOgRigEAABB6hGIAAACEnm9LsrmuG5X0hqT1xpgz/NovAAAAkGp+nim+StJSH/cHAAAA\npIUvodh13eGSPi3pdj/2BwAAAKSTX2eKfyXpGkkJn/YHAAAApE2v5xS7rnuGpM3GmIWu687sZtxs\nSbMlyRijeDze20NLkmKxmG/7CgP65Q398oZ+eUO/vKFf3tAvb+iXN7nYL8da26sduK77E0kXS2qV\nVCipTNKDxpiLuvlrtrKyslfH3S0ej6uqqsqXfYUB/fKGfnlDv7yhX97QL2/olzf0y5ts6ldFRYUk\nOT2N6/WZYmPMtZKulaRdZ4qv7iEQAwAApN3W9dtUvblGBcX5qhg7WJEoK9PiX3xbkg0AACATvTFn\nkebe/ryWvbbq49fiw8s166JjdPKXZyi/KD/A6pApfA3FxpjnJT3v5z4BAAD2hbVW5sePac7vntlr\nW9W6bfr7DY/rracX6+q7L5dya3os9gHXDQAAQE5a8PfXOg3EHa1YuEZ/+va9aaoImYxQDAAAco61\nVk/8tvtAvNvrj7+tDSs3pbgiZDpCMQAAyDmr3vpQG1YkH3Tn3f1iCqtBNiAUAwCAnFO1frun8VvW\nZsfyYkgdQjEAAMg5efne1hLIK8xLUSXIFoRiAACQc/afMkrRvGjS4yccOz6F1SAbEIoBAEDO6Tuw\nTEecflhSY0vL++jYzxyV4oqQ6QjFAAAgJ513zadVOqCkx3EXXX+e8gqYPhF2hGIAAJCTBo2K67/M\nFRo0qvMnc+QX5Wv2ry7S0WdPSXNlyEQ85hkAAOSs4QcO1Q3zv6NFz7ynlx56Qzs271RBcYEOnTle\nx37uKPXpWxx0icgQhGIAAJDTorGoppxyqKaccmjQpSCDMX0CAAAAoUcoBgAAQOgRigEAABB6hGIA\nAACEHqEYAAAAoUcoBgAAQOgRigEAABB6hGIAAACEHqEYAAAAoUcoBgAAQOgRigEAABB6hGIAAACE\nHqEYAAAAoUcoBgAAQOgRigEAABB6hGIAAACEXqy3O3Bdt1DSC5IKdu3vfmPM93u7XwAAutPa3Ko3\nn3pXa95dJ5uwGjp2kI46Y7IK+xQEXRqALNTrUCypSdLxxpha13XzJC1wXfcfxphXfNg3AAB7eeFv\nr+j+Gx7Xji01e7z+1x88pNMuO15nfuMkRSJcDAWQvF6HYmOMlVS768u8Xf/Z3u4XAIDOzL39ef31\nBw91uq2hplEP/myOdmzeqYv/9zw5jpPm6gBkKz/OFMt13aikhZLGSrrVGPOqH/sFAKCjjas2697/\nebjHcc/ctUCTTz5Uh84Yn4aqAOQCx1r/Tuq6rttP0kOSvmGMWfyJbbMlzZYkY8zhzc3NvhwzFoup\ntbXVl32FAf3yhn55Q7+8oV/exGIx/f5bd+qhm/+R1PipZ0zR9+7/Voqryly8v7yhX95kU7/y8/Ml\nqcfLRr6GYklyXfd7kuqNMT/vZpitrKz05XjxeFxVVVW+7CsM6Jc39Msb+uUN/fImHo/r0vFXadOa\n5HoWiUZ0+8qfKxqLpriyzMT7yxv65U029auiokJKIhT3+i4E13UH7jpDLNd1iySdJOn93u4XAIBP\nqq9pTHpsoi2hpnp/rkoCyH1+zCkeKumuXfOKI5KMMeZxH/YLAMAeSsv7qGZrbc8DJcUKYizPBiBp\nfqw+8Y6kyT7UAgBAt446Y7IevunJpMYeefphikRZlg1AcvhpAQDIGjMunKZYfnLnc0685LgUVwMg\nlxCKAQBZo3xoP82++aIezwC7156psYePTk9RAHKCL+sUAwCQLlPPnKw+fYtkfvKYPnx33R7bBo+O\n6+x/P1XTP3tkQNUByFaEYgBA1plw3Hgd8qkDtXrRWn24eJ0SbQlVjB2iA6ftz+OdAewTQjEAICs5\njqP9Jo3SfpNGBV0KgBzAx2kAAACEHmeKgSxjrdWKHdXa1tioolhM4/uXKz8azid2AQDgF0IxkCWs\ntXps9UqZ5R9oTc3Oj1/vm1+gM8fsp4vHH6I+eXkBVggAyCY7q2r09jNLVL+jQcV9izTpxAkqG1AS\ndFmBIRQjY1XW1erRVSu0dPs2JazVyNIynTlmf43vXx50aWlnrdUNC1/TE2tW7bVtR3OT7v5gqV7d\ntEE3H3eCyvLzA6gQAJAtarfX6Z7vP6hXH3tLbS1tH78ey4tq6tlTdOH3z1VJ/z4BVhgMQjEyTptN\n6JZFb+n+FctkO7z+1pbNemTVCk0bUqHvT52mkrzwhL/7VyzrNBB3tLy6Wj9+/RXdMJ0HFgAAOlez\nrVY/+syvtWHFpr22tba06Z/3v67V73yk6x64MnTBmBvtkHFuemuh/v6JQNzRyxsr9e0F89Xc1tbF\niNzSZhP62/IPkhq7YMN6re0wtQLebayv022L39EVz8/T7Gef0vde+ade3lCphO3qHQkA2eMv193f\naSDuqHLZRt393QfSVFHm4EwxMsoH27fp4VUrehz37tYqPfnhap2131hfj9+SaNOWhgZZKw0sKvJ1\n3/vqnaoqbayvS3r8U2vX6P8dMjGFFeUma63ueG+x7lq6WIkOr7+3baueXbdW4/r100+OOU5DisN1\n5gT+SLQltOrttaqtrlNxaaH2mzQq6cdVA37ZtqFab8xZlNTYVx9/Sxd87xz1G1SW4qoyB9+RyCgP\nrVye/NhVy30LxVsa6mWWf6An1qzSzuZmSVJJXp4+O/4QnTliZKBBaEtDvcfxDSmqJLfd8d5i/Xnp\n4i63L6+u1lXzn9Vtx5+svgUFaawM2ayttU1zb5+vZ+58UVXrtn38et+BpZp54TH69BUnqqAoPFPB\nEKw3576rRFui54GSEq0JvTn3HR1/8bEpripzMH0CGeXtqs1Jj11eXa36lpZeH3N59XZdOu9J3bvs\n/Y8DsSTVtrTornff1qXzntTSbVt7fZx9VeBxuTWv4yFtrq/XXe8v6XHc+rpa3btsaRoqQi5obWnT\nb2bfob/97yN7BGJJ2rGlRo/cPFc3XnCrGuuaAqoQYVO7Pfmrju3jvZ2UyXaEYmSU5iQ/we7WlOjd\nvOLalmZdveB5bW/q+pfSzuZmXb1gvrY3NfbqWPtqYnyg8jw8tvbwQYNTWE1uemz1yqTnDD++ZpVa\nevm+Qzg8fNOTeuuprq8+SNKKhWv0f9+9P00VIeyKSwu9jS/LjGmE6UIoRkbxMk2hKBZTaS9XoJiz\nZrW2NvYcdnc0N+nx1St7dax91b+gULOGj0hq7KCiYk0fOizFFeWed6q2JD22uqlJH9XUpLAa5IKm\nhmY9e9eCpMa+/NBCVW/mBlmk3qSTJshxnKTGOhFHk048JMUVZRZCMTLK6aPHJD321FFjFPNwBrUz\nT6xJPug+trr7JdFS6asTDlO8sPtP7BHH0benHNnrnoRRs8czv80Jb1c0ED7vPPOe6nYkd+m5raVN\nrz36ZoorAqRBo+I67ISDkxo7+aQJig8P13MB+O2JjHLiiFEalMSqD3mRiM7b/4BeH299bW3SYzfU\n1Qa2LNfg4j66ZeYJGtu3X6fb+xUU6CfTPqVpQyvSXFluqOiT/BOcIpIGFxenrhjkhO0bq72N38SZ\n4nSqq67X039+Qf/33/frnu8/qJcefEPNjc09/8Uc8MUfuyof2vnvkt0GDOuvL/zoc2mqKHOw+gQy\nSmEsphunz9A3X3xO1V3M882LRPSDqcdoVFnvl4mJRSJSkusdRxxHyV10So3hJaX684mn6q0tm/X0\nRx9qW2OjimMxTR0yVLOGj+QGu144ffQYzV27Jqmx0yuGqX+Bt3l5CJ/8Ym8rlOQX8Yj2dEi0JXT/\nTx/X03e8oObGPW/UvveHD+mz/3mGZl44LaDq0qO8op/++5Fv6s/fvk/vzn9/r+0TZx2kL914gfoP\n6RtAdcEiFCPjjO3XX7efcIr++sFSPfnhatW3tkqSoo6jGcNG6MIDD/LtUc+HlMf16qYNSY2dMCCe\n9FysVHEcR1MGDdYUbqbz1ZSBg3VQ/wFaur37VUYiki444KD0FIWsdvD0cXIcRzbJq0uHHNv7K1/o\nnrVWf/z3e/TSg290un3n1lr9+Zr71FjbqFNnz0pzdek1oKK/rr7ncm1ctVkLn3xXdTvqVdKvWIef\nOlGDxwwMurzAEIqRkYYU99F/TD5Clx86SWtrdiphrYb2KVE/n9eHPXf/sUmH4nP3H+frsZE5HMfR\nT445Vle98Jw+7OKJgBFJ1xx+lA6Lh/cXxicl2hJ6+5klenveEjXWNqq0vERTz5qscUfuF/gHyKAN\nGhXXoTPH653nel7Cb8RBFRp35H5pqCrcFj75TpeBuKP7/vcRTT75UA0eHU9DVcEast8gffprJwRd\nRsYgFCOjFcViOtCns8KdmTa0QkcNHqLXNm3sdtyk+EDNHJbcChDITvGiYv1+1kkyyz/Qo6tXamtj\n+0NQIpKOrRiuCw4Yr4kE4o8te22l/nDl3Xutvzvvzhc16tDh+vpvLwn1GSdJ+rcffkYr37pJddVd\n33CXVxDTJTecH/oPEenwTJKrgdiE1fN3/1Pn//fZKa4ImcZJ9tKOz2xlZaUvO4rH46qqqvJlX2FA\nv/ZW39qiH776sv65YX2n248cNETXT5uukl4u/xYGufL+ak0ktL62Vs2JNg0qKk7ZE+yytV/L31it\nn55/i1qaWrsc03dgqb732H/4evd6NvZr3fuVuvXyO1W5fNNe2wYM66/LfnOxDjhq/5QcOxv7lSrN\nDc36yrhvJz2+Ytxg/eS576SwouyXTe+viooKST3fFsSZYoRecSxPNxzzKS3ZtlUPr1qulTuqZa00\npqyvvjD5cI3Oy+/2LE5VQ4Ne2VipupYWlRUUaPrQCpXl8xjgbBaLRHy5kTMXWWt1x7fv7TYQS+1P\nbLvv+kd0xR++lKbKMtPw8RX60TP/pfcWLNNrj72t2u11Kiot1OSTJ2jySRMUjXGDbDo01ntbWaKh\nJpiHNSFYoQnFrYmEtjY2yJGjAUWFijqsRod/cRxHEwbENWHAnnPIuvskvK2xQTcvelPPr/tIbR2u\nuORHojpl1Gh9feIkzi4j57z/0opOz3p2ZuGT72j7xh2hvIu9o0gkognHjdeE48YHXUpoFZcWKpoX\nVVtLcqsNlcVLU1wRMlHOh+KN9XX6+/IPNGfNatW0tH9SLC8o1Jlj9td5Yw9Q/0KWVoJ3WxsbdPlz\n81RZt/c6x82JNj22eqXe375Vv5lxAsEYOeXdF/ZewqkribaElr60XMd85ogUVgT0LJYf0xGnTdSr\nj76V1PipZ01JcUXIRDl9unTJ1ipdOu9J/W35Bx8HYkna1tSou95fokufeVJrdu4IsEJkqxsXvt5p\nIO5oeXW1bn3n7TRVBKRHs8fL0E31na83DqTbSZfOSGpcQZ8CHXfB1BRXg0yUs6F4W2ODvv3P+drZ\n3PUP8C0NDbp6wXzVt7Z0OQb4pPW1NXqpi5vyPmnuh2u0s5lQgNzRd7C3udb9Bod76gQyx7gjxuj8\n687qdkwsL6qv//aLKi1P/imXyB29nj7huu4ISX+RNFiSlXSbMebm3u63tx5etaLbQLzbxvo6PbX2\nQ52z39g0VIVcMH/9OiW7Zktzok3/3FCp00aNSWlNQLocffYUPfDTJ5J6KEVpeR/m0SKjnH75CSqv\n6K9Hbp6rymV7LsU54djxOufqUzXuCH5eh5Ufc4pbJX3LGPOm67qlkha6rvu0MeY9H/a9zx5fvSrp\nsU+sXkkoRtJ2eDzzu7OLx1UD2WjgiAE64vTD9PoTPU8NOuGLn1JeQc7fuoIsc/TZUzT1rMlasXCN\nNqzcpEg0ojETR+qwYyZkzRJjSI1e/7QyxmyQtGHXn2tc110qaZikwEJxS6JNmxu6Xiz9kz6qrUlh\nNeiMtVbNiYTyI5GsW7S+JC/P0/g+HscDme5LN56vTWu2aO2SrqcRTTnlUJ111clprApInuM4GnfE\nGM4KYw++foR3XXe0pMmSXvVzv15FPIasWCRnp1ZnnHeqtuiBlcv04vr1ak60qSAa1cxhI/TZsQfo\n4PIBQZeXlOlDh+sPi99JamzUcTRtaEWKKwLSq0/fYn3ngSv1yE1z9cJ9r6hux79OQpRX9NOJlxyn\nU2fPZA1eAFnFtyfaua5bImm+pB8ZYx7sZPtsSbMlyRhzeHMS832TEYvF1Nq69yLyn3vgb3p3y+ak\n9jFz1Gj9/rQzfakn03XVr1Sz1urm11/R79/s+rnz3zzqaF025cg0VtWzrvr1xUcf0quV63r8+6eP\nHadfnnhqKkrLSEG9v7JVLvSrqaFZS19epvqaBvWNl2n80eMUjabmREMu9Cud6Jc39MubbOpXfn6+\nlMQT7XwJxa7r5kl6XNJcY8wvk/grKX/M8xNrVuknbyR3wvpn02eE5mxeUI9lfGDFMt309sIex/3X\n4UfpjDGpeeTpvuiqX+tra3T5c/O0ranrpx4N61Oi3806UeWFRaksMaNk02M/MwH98oZ+eUO/vKFf\n3mRTv5J9zHOvP867rutI+pOkpUkG4rQ4acQoHdS/58vxRw8ZqqlDhqahovBqSbTpzqWLkxr756WL\n1WYTvTpedVOTzPIP9PM3X9ev3l6oJz9craa25J5ilKxhJaX63ayTNGXgoL22OZKmDx2m34YsEAMA\nkM38mFM8XdLFkt51XXf37cjfMcbM8WHf+yw/GtXPj52h77z8ohZVbel0zPShw/SDqcd4noMMbxZU\nrtf2JFdg2FRfr9c2btynM/etiYR+9+7benDlcrUk9gzWv1n0pr464TCd5eMqI8NKSvTrGSdo9c4d\nWlC5TrUtLeqbX6AZw0ZoWAlrXAIAkE38WH1igZI4JR2EvgUF+s2ME7Rw80Y9smql1tTskCNHB/Tr\nr7P3G6tDB8SzbuWDbLRm505P41fv3OE5FCes1fWvv6xnPlrb6fYdzc268c3X1dDaqvMP8Hfd1DFl\nfTWmjAcUAACQzXJ+AcmI4+jIwUN15GCmSATF68eOffmc8vz6j7oMxB3d+s5b+tSw4arow5lcAADw\nL6xFhpQb26+/p/Hj+nobL0kPrlie1LiEpEdWrfC8fwAAkNty/kxxrlpevV1z1qzSxvo65UWiOiw+\nUKeMGq2SvPygS9vL0UOGalBRcVIPVBlRUqopgwZ72n99a4verkpu+T1JemlDpS4/dJKnYwAAgNxG\nKM4yO5qa9MPXXtJrm/Z8Zvuz69bqd+8u0tcnTtK5+48LqLrOxSIRfXXCRF3/+is9jv3qhMM83/jY\n4HGdxPrWFk/jAQBA7iMUZ5H61hb9+4vPaVn19k63N7a16hdvvaGEtfrs2APSXF33Thk1Rjubm/Wb\nRW+qswXXIo6jqycfoZnDR3jed0levqKOo7Yk19wuLyj0fAwAAJDbCMVZxCz/oMtA3NEt77yl44eP\nVP/CzAp/nxt3oKYOGaqHV67QC5XrVNPcrLL8fM0cPkLn7DdWw0pK92m/BdGoZgwboWfX9XyjnSSd\nOGLUPh0HAADkLkJxlmhNJPToqpVJjW1JJPT4mlW6ePzBKa7Ku5GlZbpy0hRdOWmKr/s9b+wBSYXi\nPrE8nTZ6jK/HBgAA2Y/VJ7LEutqapG5U2+2NzRt7HpRDJsYH6ms93DyXF4no+qOnqyy/IE1VAQCA\nbMGZ4izR6PExxc0+P9Y4G1x44EEaXNxHdy5drNU7d+yx7fCBgzV7wkQdMiAeUHUAACCTEYqzRLyw\nyNv4Im/jc8UJI0bq+OEjtGTbVn1UW6Oo42h8/3KNLC0LujQAAJDBCMVZIl5UpCMHDdHrSU6LOHVk\neOfNOo6jCQPimsBZYQAAkCTmFGeRCw4Yn9S4UaVlOnooj7UGAABIFqE4i0wdMlSXTTis2zHxwiLd\ncMxxijr80wIAACSL6RNZ5qLxB2tUWZnueX+pFm+r+vj1wmhMJ48cpS8dPEEDi4oDrBAAACD7EIqz\n0KcqhutTFcO1tmanNtTVKS8S0YH9y9UnLy/o0gAgJay1WvTcEr06d6Fam1s1YFh/TT1rikr69wm6\nNAA5glCcxUaWlrGqAoCc994/l+kv192vDSs27fH6X3/4sGZeOE0XfPcc5RXw6wxA7/BTBACQsRY9\n+55+dekflWhN7LWttblV8+58UZs/rNJVd3xFsbxoABUCyBXcjQUAyEhNDc36w5X/12kg7uid55bq\n2b8sSFNVAHIVoRgAkJFeeXih6qqTe7z9M3ctkLU2xRUByGVMnwAAZKQ3n1qc9NiNqzZrw4pNqhg3\nJIUVQZIqV2zS2iXrZa1VxbjBGnXI8KBLAnxBKAYAZKSGnQ2extfvbExRJZCkZa+t1AM3ztH7r6zY\n4/Uxh43Uuf9xqg474ZCAKgP8wfQJAEBG8rrcWmk5y7Olyptz39UN7q17BWJJWr1orW665I+af+/L\nAVQG+IdQDADISEeeMSnpsSMPGaZBo+MprCa8tm/cod9+/S61tbZ1OcZaqzv/y2jd+5VprAzwF6EY\nAJCRjjz9MPUb3DepsSddepwcx0lxReH0/D0vqaWxpcdxibaE5t35YhoqAlKDUAwAyEix/Ji+cduX\nVFCc3+246ecdqU+5U9NUVfi88sibSY99+eGFrAKCrEUoBgBkrLGHj9F1D12l8dPG7rWtdECJzvvP\nT+v//fJCzhKn0M6qmqTHNtY2JXVWGchErD4BAMhoow4Zrmv//g3VVzXp1SffUEtjiwYML9fEWQfz\neOc0KOxToPokVwKJxiKK8W+CLMU7FwCQFUaOH6bieEHQZYTOobMO0vy/JreyxKGzDlYkwkVoZCdf\nQrHrundIOkPSZmPMBD/2CQCAnxJtCb39zBLN/+vLqly+SU7E0ehDh2vWRdM1ftpYpmB04YQvHpt0\nKD7hC8emuBogdfw6U3ynpFsk/cWn/QEA4JsdW3bqpkv+qNWL1u7x+qbVW/Tqo2/psOMP1td+d4kK\n+3Am+pNGHTJcZ155sh779VPdjjvu/Kk6dOb4NFUF+M+XaxzGmBckbfNjXwAA+Km5oVk/v+j3ewXi\njhY9+55u+eqflUgk0lhZ9vjst0/X+dedpcKSvT805BXE9Omvn6gv3XgBZ9uR1dI2p9h13dmSZkuS\nMUbxuD+LrMdiMd/2FQb0yxv65Q398oZ+ebOv/Zrzx3lau2R9j+PefX6p1r69QUecfNi+lJdx/H5/\nfeG75+u8b56l+fe9pJWL1shaaeRBwzTr89NVNqDUt+MEhe9Hb3KxX2kLxcaY2yTdtutLW1VV5ct+\n4/G4/NpXGNAvb+iXN/TLG/rlzb7267HfdX/Zv6OHb5mj0VOGeT5GJkrV++vIcw/Tkef+64NDs21S\nVVWT78dJN74fvcmmflVUVCQ1jltEAQA5q6WpVWvf6/ks8W6r3u56igWA3EYoBgDkLK9zhBNtzCkG\nwsqXUOy67r2SXpZ0oOu661zX/bIf+wUAoDfyC/PUf0jfpMcPHjMwhdUAyGS+zCk2xnzej/0AAOAn\nx3E04/PT9PBNTyY1fuaF01JcEYBMxfQJAEBOm3XxdJX079PjuMGj45p65uQ0VAQgExGKAQA5rd+g\nMv3HX77abTCOjyjXt+6+XPlF+WmsDEAmSduSbAAABGX/yaN0/VPXaN6dL+qFe19WzbY6SVJ5RT8d\nf/F0zbooubPJAHIXoRgAEArlQ/vJvfZMnXfNp1WzrVZOJKKS/sWKRLhoCoBQDAAImUg0or4Dy4Iu\nA0CG4eMxAAAAQo9QDAAAgNAjFAMAACD0CMUAAAAIPUIxAAAAQo9QDAAAgNAjFAMAACD0CMUAAAAI\nPUIxAAAAQo9QDAAAgNAjFAMAACD0CMUAAAAIPUIxAAAAQo9QDAAAgNAjFAMAACD0CMUAAAAIPUIx\nAAAAQo9QDAAAgNAjFAMAACD0CMUAAAAIPUIxAAAAQo9QDAAAgNCL+bET13VPlXSzpKik240xN/ix\nXwAAACAden2m2HXdqKRbJZ0m6WBJn3dd9+De7hcAAABIFz+mTxwlaYUxZpUxplnSfZLO9mG/SIGW\ntjZtrq9bsZv0AAAX8UlEQVTXloZ6tSYSQZcDAACQEfyYPjFM0kcdvl4naaoP+4WPNtfX6+8rPtCc\nD1drR1OTJKl/QYHOHDNW540dp/LCooArBAAACI4vc4qT4brubEmzJckYo3g87st+Y7GYb/vKVe9u\n3qSvPDNX1U2Ne7y+valJf3l/iZ5cu0Z3nHGOxpaXB1Rh5uL95Q398oZ+eUO/vKFf3tAvb3KxX36E\n4vWSRnT4eviu1/ZgjLlN0m27vrRVVVU+HFqKx+Pya1+5aHtTo77y1BxV7zo73JnN9XW69LGHdPfJ\np6s4Ly+N1WU+3l/e0C9v6Jc39Msb+uUN/fImm/pVUVGR1Dg/QvHrksa5rjtG7WH4AkkX+rBf+ODR\nVSu7DcS7bW6o19y1a3Tu/uPSUBUAAEBm6fWNdsaYVklXSJoraWn7S2ZJb/cLfzy+emXSYx/zMBYA\nACCX+DKn2BgzR9IcP/YF/7QmEtpQX5f0+HW1NSmsBgAAIHPxRLscFnEcReQkPT4W4e0AAADCiRSU\nwyKOo4M8rChxcPmAFFYDAACQuQjFOe4cDzfOnbsfN9kBAIBwIhTnuBNHjNQhSZwBPmrwEB09dGga\nKgIAAMg8hOIclxeJ6mfHztCk+KAux0wbUqH/nXasog5vBwAAEE5pe6IdglOWX6BfzzheCzdv0pPr\nP9IHWzZLjjSub3+ds99YTYwPlOMkf0MeAABAriEUh0TEcXTk4CE67ZAJWfMEGgAAgHThejkAAABC\nj1AMAACA0CMUAwAAIPQIxQAAAAg9QjEAAABCj1AMAACA0CMUAwAAIPQIxQAAAAg9QjEAAABCj1AM\nAACA0CMUAwAAIPQIxQAAAAg9QjEAAABCj1AMAACA0CMUAwAAIPQIxQAAAAg9QjEAAABCj1AMAACA\n0CMUAwAAIPQIxQAAAAg9QjEAAABCL9abv+y67uck/UDSQZKOMsa84UdRAAAAQDr19kzxYkmfkfSC\nD7UAAAAAgejVmWJjzFJJcl3Xn2oAAADSok0xrVREtUqoVLLlQReEgPUqFAMAAGSXFhXrARU7Dyvm\nbPz4VbtjpIp1lup1johH4dTjv7rruvMkDelk03XGmEeSPZDrurMlzZYkY4zi8XjSRXYnFov5tq8w\noF/e0C9v6Jc39Msb+uUN/eqEbVak5gpFWl/aa5OTWKuyyC0qyXtPiZJfSE5eAAVmj1x8fznW2l7v\nxHXd5yVd7eFGO1tZWdnr40pSPB5XVVWVL/sKA/rlDf3yhn55Q7+8oV/e0K+9lTq/Vh/nQVkrOc7e\n23e/XmsvVK2dnf4Cs0g2vb8qKiokqZN/8T2xJBsAAMh5jmpUrCfa/9xFPNr9erEekaOGNFWGTNGr\nUOy67rmu666TNE3SE67rzvWnLAAAAP8Uar4cpympsRGnTgXae4oFcltvV594SNJDPtUCAEAGscrT\nOyp2HlNMKyVZtWo/Ndgz1axJSuJqLDJI1Nnsbby8jUf24/ZKAAA+wVGd+jk/VIHz2h6v52mNipxn\n1WSPULX9gaxKAqoQXllb4OlzjFVB6ooJAUf1KtTTKnTmK6IdsipRo52uBp0qq7Kgy+sUoRgAgD20\nqZ/zXRU4b3Z6Q5a1UoHzhvrrOm2zvxC/SrNDsyZLUpc32e22e3v71QDsi3y9oX7ODxVxavZ83Vmk\nEvsn7bDXqkkzgymuG3wnAwDQQYEWdBmIpfbXrG3/BV9oX1SjZqW/yAA4qlaRnlSes1qSVasdowad\nqoT6B11aUlp0kFrsAcpzlnU7znGkZjtRrdovTZXlljwtUX/nWjlOS6fbI06T+ul/VG0L1KRpaa6u\ne4RiAAA6KHbal+Dv7mzix6sUOA+r0eZ6KE6oxPmT+sjsGXQcqcTeoTp9VrX2K5KigVWYHEc77TdV\nrm/KcZq7HJWwRdppv5HGunJLqfPbLgPxbo6TUKl+oyY7VZm0EFrmVAIAQAbI03spGZutypybVeLc\n00XQaVGJc5/KnF9I6v1zD1KtRQdrm/252mznD51os4O13f5SrRqX5spyQ0zLle8sUTKPwIg5lcrX\nwtQX5QFnigEA6MBRm4fRbWoPg7m5EkX76huP9DiVpNiZo0Z7opo1Jf1FetSiidpi71OBXaBC50VF\nVKOEypRfeqa27DxERKN9l6clkrq/ytJRvrNYzfbIFFbkDf/yAAB00KphytOapMa2aZhyNRBLUrHT\nvupqslNJmm3mh+J2MTVppprszI9fiefHJWXHE9oylaPup03srTUldewrpk8AANBBgz09JWOzUYFe\nT3psvoexyE3tHxI9jLdDU1TJviEUAwDQQYNOU5sd0OO4Nttf9crtUOyo0ePYzJ9XjNRp0lFqs+VJ\njU3YwoxbuYVQDABAB1al2m5vUJvt1+WYNttP2+1PZdU3jZWlX0I9fzj419hy5fJUEiQjpjr7eUnq\n8ma73a/X61xZ9UlTXckhFAMA8AmtGqet9nbV2s8rYf/19K2ELVOtvUBb7R/VqgMCrDA9GnSih7En\npbASZIt6nac6e87Hc813h+Dd/3ccqcHOUq39cjAFdoMb7QAA6ERCcdXar6pWX1bEbpNkd505Dc+v\nznp7lvro792u6ytJ1uapwZ6dpqqQ2RzV2KvUbCerj/Og8p1F7a86UrMdr3p7jhp1sjLxvGx4vrMB\nANgnMSU0KOgiApHQYFXb76qffijHad1jabbdf7Y2qmp7ndqUWTdNIUiOmjRDTXaGIna7HO2UVR8l\n1Pn60Jki82I6AADIGE36lLbZX6nJHrHH0myOIzXZKdpmb1KTZgZWHzJbQv3VplEZH4glzhQDAIAe\ntGiCttufK2rXK6bVkqxaNUZtGh50aYBvCMUAACApbRrmeS1aIFswfQIAAAChRygGAABA6BGKAQAA\nEHqEYgAAAIQeoRgAAAChRygGAABA6BGKAQAAEHqEYgAAAIQeoRgAAAChRygGAABA6BGKAQAAEHqx\n3vxl13V/JulMSc2SVkr6kjGm2o/CAACAP6LaoCLnCcX0oSRHLXacGnS6EhoQdGlAxujtmeKnJU0w\nxkyUtEzStb0vCQAA+KNZZc6NijsXqsS5W4XOiyp0XlBp5E8a6Lgqcf4gKRF0kUBG6NWZYmPMUx2+\nfEXSeb0rBwAA+MIm1M+5XoXOi7K2swFtKnHulaMG1dhvprs6IOP4Oaf4Ukn/8HF/AABgHzktz3wc\niB2nk+2OZK3Ux3lYeXov/QUCGabHM8Wu686TNKSTTdcZYx7ZNeY6Sa2S7ulmP7MlzZYkY4zi8fg+\nFfxJsVjMt32FAf3yhn55Q7+8oV/e0C9vojVGUueBeLfd2/oX/kOJkuPSUFXm4v3lTS72y7GdX1NJ\nmuu6l0j6qqQTjDH1Sf41W1lZ2avj7haPx1VVVeXLvsKAfnlDv7yhX97QL2+896tRRXpORc4cRbVB\nVjG1aILq7dlq0QRJ3aTFrNemwZGT5CQ5X7jNxrXF3p/imjIb34/eZFO/KioqpCS+4Xu7+sSpkq6R\nNMNDIAYAIKWi+lD9nf9UzNm4x+sxbVSRM08N9kTtsP8pKS+YAlOuJelALEmOGlNYC5Adejun+BZJ\npZKedl33bdd1f+9DTQAA7LOItqrc+dZegXg3a6UiZ57KnF+kubJ0KpB1ypIe3abcugwO7Iverj4x\n1q9CAADwQ7Hzd0Wdri/r7r7BrNh5UvXWVav2S2N16eLI5p8pp6nLW3320GhPSXE9QObjiXYAgBzS\nomLN6XHU7hvMipxHU1xPcBKFF8rans99JWyJ6nVaGioCMhuhGACQM6LaqIizM+nxefoghdUELDpK\n1fa/ZW1UkvZYq3j3nxO2UNX2eln1C6BAILP0avoEAACZxdvT2bzcjJaNmjRT2+xA9dH/qUCvStqd\njKNqsJ9Snf1Cjk4fAbwjFAMAckZCg2RtgRynKanxrRqR4oqC16JDVG1vUESbFLMfSXLUqtFKaEDQ\npQEZhVAMAMgZVkVq0Ak9zive/ZS3entmmioLXkKD1azBQZcBZCzmFAMAckqdPV8JW9DtGMeRmuwk\ntWhimqoCkOkIxQCAnNKmUaq2P1LCFu7xescbzZrtQaq2P1RuP9UOgBdMnwAA5JxmHaGt9s8q1kMq\n0j8UcWrkOFKL3V/19iw16FRJ3Z9NBhAuhGIAQE5q01DV2K+pRpfJsXWSorIqDrosABmKUAwAyHER\nWZUGXQSADMecYgAAAIQeoRgAAAChRygGAABA6BGKAQAAEHqEYgAAAIQeoRgAAAChRygGAABA6BGK\nAQAAEHqEYgAAAIQeoRgAAAChRygGAABA6BGKAQAAEHqEYgAAAIQeoRgAAAChFwu6AADAvolou4o0\nR0XOU4qoSlYFatbhqrfnqEWHBF0eAGQVQjEAZKF8val+zncVceo6vFqnIj2tIudp1duztdNeKSka\nVInIKG3K10LFtE5WUbXoYLVqXNBFARmFUAwAWSamlernXKuI09TpdmulYucRJVSsWvvVNFeHzGJV\npMdV4tytqLNpjy3N9iDV2MvUosMCqg3ILMwpBoAsU+Lc1WUgliTHaQ/GfWQU0dY0VoZMU+Lcrr6R\nX+wViK2V8p2lKne+pQK9FFB1QGbp1Zli13Wvl3S2pISkzZIuMcZU+lEYAGBvEW1VgRbI2vbw25X2\nbW0qsv9QnS5KV3nIIE7LP1Xi3NPpe2X3147Tqr66Xlvs32RVlv4igQzS2zPFPzPGTDTGTJL0uKTv\n+VATAKALMa2W4yS6DcQd5TnLU1sQMpbTeE/7/3t4r0ScBhXpH74fP6IqFWquivSwCvSCpEbfjwH4\nqVdnio0xOzt82UeS7V05AIDuef0xy4/lMHJUL6flxR6vKOxW6Dyvenu+L8eOaLPKnFtVoBflOImP\nX0/YEtXrLNXaL0nK8+VYgJ96faOd67o/kvQFSTskzep1RQCALrVqpKx1JNmkwk6rRqe6JGQgR7Vy\nZKUkryhEtMOX40a1QeXONxR1qmQ/8Xks4tSqRH9VnpZru/2xCMbINI795Lv2E1zXnSdpSCebrjPG\nPNJh3LWSCo0x3+9iP7MlzZYkY8zhzc3N+1x0R7FYTK2trb7sKwzolzf0yxv65c2+9itSc7kiLS92\nO6b9J7ujtr5zpWjFPtWXaXh/eZCoUax6mqySy8U2epDa+v6914eN7vg3OW2Luj7Ornrair4mW/S1\nXh/PT7y/vMmmfuXn50tJfCv0GIqT5bruSElzjDETkhhuKyv9uR8vHo+rqqrKl32FAf3yhn55Q7+8\n2dd+5WmJyp0r5ThtnW7ffcm83p6hnfbq3paZMXh/eTM4/2o5rW8kNbYmcYnqdEmvjhfT+4pHLktq\nykab7a8t1iiTzhbz/vImm/pVUVEhJRGKe3Wjneu6HVf+PlvS+73ZHwCgZy06RNX2e7K280DhOFKj\nPU477VVprgyZJFF4oSTtNY3hk6yNqUFn9Pp4Rc6zkpKbwxx1titfXZ9RBoLQ2znFN7iue6Dal2T7\nUNJlvS8JANCTJs1Qld1fxXpYRXpKEWenrI2oWVNUb89Rk44RS9GHm807SQ32FBU5c7sdt9P+hxKK\n9/p4Xucl+zWPGfBLb1ef+KxfhQAAvGnTcNXYK1SjKyTbrPYf6QRh7OI42mGvUZsGqVj3K+I07LG5\nzQ5Sjb1cjT7dI59Qn5SOB1KNxzwDQE7ID7oAZKSoau2XVafPq9A+q6izXlJULfYgNWmq/IwBTfYY\n9XEeTGpswharRRN9OzbgB0IxAAA5zqq4fd5wCpetbtYUtdoRijkf9Ti2QafKqjh1xQD7gOtsAADA\nBxFV2+uUsIWS9r7Bb/fXLXY/1dpL01wb0DNCMQAA8EWrxmubvVktdv9OVqFw1GiP0zb7K1mVBFEe\n0C2mTwAAAN+06kBttbcrzy5RgfOaHNUrYcvVqFlq09CgywO6RCgGAAA+c9SiCWqxyTzPC8gMTJ8A\nAABA6BGKAQAAEHqEYgAAAIQeoRgAAAChx412AIBQiWizijRPEWeLrArVbCerWUeI80RAuBGKAQCh\n4KheZc4vVahn5TiJDhvuVasdpp32W2rWlOAKBBAoPhYDAEKgUf2dq1XkzJOU2GtrzFmv/s41ytfr\n6S8NQEYgFAMAcl4f3ad85z1J6uRJa9r1eqv6Oj+W1Jy+wgBkDEIxACDHtarYeTSpkVFnuwr1Qorr\nAZCJCMUAgJyWp2WKOtuSHl/gvJTCagBkKkIxACCnOapP6XgAuYFQDADIaQn19TTeehwPIDcQigEA\nOa1VY9VqRyQ9vsEen8JqAGQqQjEAIMc5qrPnSZKs7X5kqx2hZh2ZhpoAZBpCMQAg5zXoTDXYEztd\njm13UE7YMlXb/xG/GoFw4jsfABACEe2w16omMVtttnyvbY32WG21v1WrxgRSHYDg8ZhnAEBIRFWn\nC1VnXeXbhYpqi6zy1axJSmhQ0MUBCBihGAAQMjE1a2rQRQDIMEyfAAAAQOgRigEAABB6hGIAAACE\nHqEYAAAAoefLjXau635L0s8lDTTGVPmxTwAAACBden2m2HXdEZJOlrS29+UAAAAA6efH9ImbJF0j\nqYeHZwIAAACZqVeh2HXdsyWtN8Ys8qkeAAAAIO16nFPsuu48SUM62XSdpO+ofepEj1zXnS1ptiQZ\nY1RRUeGhzO75ua8woF/e0C9v6Jc39Msb+uUN/fKGfnmTa/1yrN23WQ+u6x4q6RlJ9bteGi6pUtJR\nxpiN/pSXVB1vGGOOSNfxsh398oZ+eUO/vKFf3tAvb+iXN/TLm1zs1z6vPmGMeVf618PiXdddI+kI\nVp8AAABAtmGdYgAAAISeL+sUS5IxZrRf+/LotoCOm63olzf0yxv65Q398oZ+eUO/vKFf3uRcv/Z5\nTjEAAACQK5g+AQAAgNDzbfpE0HjUdPJc171e0tmSEpI2S7rEGFMZbFWZy3Xdn0k6U1KzpJWSvmSM\nqQ62qszluu7nJP1A0kFqX43mjWAryjyu654q6WZJUUm3G2NuCLikjOa67h2SzpC02RgzIeh6Mtmu\np8z+RdJgtT9U6zZjzM3BVpW5XNctlPSCpAK1Z6L7jTHfD7aqzOe6blTSG2p/VsUZQdfjl5w4U8yj\npj37mTFmojFmkqTHJX0v6IIy3NOSJhhjJkpaJunagOvJdIslfUbtv2jwCbt+mdwq6TRJB0v6vOu6\nBwdbVca7U9KpQReRJVolfcsYc7CkoyV9nfdXt5okHW+MOUzSJEmnuq57dMA1ZYOrJC0Nugi/5UQo\nFo+a9sQYs7PDl31E37pljHnKGNO668tX1L4mN7pgjFlqjPkg6Doy2FGSVhhjVhljmiXdp/YrN+iC\nMeYFSduCriMbGGM2GGPe3PXnGrUHl2HBVpW5jDHWGFO768u8Xf/xO7EbrusOl/RpSbcHXYvfsn76\nRMdHTbuuG3Q5WcN13R9J+oKkHZJmBVxONrlU0t+CLgJZbZikjzp8vU7S1IBqQQ5zXXe0pMmSXg24\nlIy26+rNQkljJd1qjKFf3fuV2k9ElgZdiN+yIhT79ajpMOmuZ8aYR4wx10m6znXdayVdISnUc6h6\n6teuMdep/dLkPemsLRMl0y8AwXFdt0TSA5K++Ymrg/gEY0ybpEmu6/aT9JDruhOMMYuDrisTua67\ne27/Qtd1ZwZdj9+yIhQbY07s7PVdj5oeI2n3WeLhkt50XTetj5rORF31rBP3SJqjkIfinvrluu4l\nar/R5wRjTOgvrXl4f2Fv6yWN6PD18F2vAb5wXTdP7YH4HmPMg0HXky2MMdWu6z6n9vnrhOLOTZd0\nluu6p0sqlFTmuu7dxpiLAq7LF1kRirvCo6b3jeu644wxy3d9ebak94OsJ9PtWingGkkzjDH1QdeD\nrPe6pHGu645Rexi+QNKFwZaEXOG6riPpT5KWGmN+GXQ9mc513YGSWnYF4iJJJ0n6acBlZSxjzLXa\ndbP5rjPFV+dKIJayPBRjn93guu6Bal+S7UNJlwVcT6a7Re3L9Ty964rEK8YYetYF13XPlfQbSQMl\nPeG67tvGmFMCLitjGGNaXde9QtJctS/JdocxZknAZWU013XvlTRTUtx13XWSvm+M+VOwVWWs6ZIu\nlvSu67pv73rtO8aYOQHWlMmGSrpr17ziiCRjjHk84JoQEJ5oBwAAgNDLlSXZAAAAgH1GKAYAAEDo\nEYoBAAAQeoRiAAAAhB6hGAAAAKFHKAYAAEDoEYoBAAAQeoRiAAAAhN7/B5WUIZ/uyrTPAAAAAElF\nTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x1173db550>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"X = np.vstack([np.random.randn(10,2) + [2,2],\n",
" np.random.randn(10,2) + [-2,2],\n",
" np.random.randn(10,2) + [2,-2]])\n",
"Y = np.repeat(np.eye(3), 10, 0)\n",
"\n",
"from matplotlib import pyplot as plt\n",
"plt.style.use('ggplot')\n",
"%matplotlib inline\n",
"import matplotlib\n",
"matplotlib.rcParams['figure.figsize'] = (12,6)\n",
"plt.scatter(X[:,0], X[:,1], 100, np.argmax(Y, 1), lw=2, cmap=plt.cm.viridis)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>LinearMulticlass/W</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>(none)</td>\n",
" <td>True</td>\n",
" <td>(2, 3)</td>\n",
" <td>True</td>\n",
" <td>[[-1.86126389328, 0.274590690509, -0.749392536...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>LinearMulticlass/b</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>(none)</td>\n",
" <td>True</td>\n",
" <td>(3,)</td>\n",
" <td>True</td>\n",
" <td>[0.898180439959, -0.0512106496384, -0.19183247...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape fixed_shape \\\n",
"LinearMulticlass/W Parameter None (none) True (2, 3) True \n",
"LinearMulticlass/b Parameter None (none) True (3,) True \n",
"\n",
" value \n",
"LinearMulticlass/W [[-1.86126389328, 0.274590690509, -0.749392536... \n",
"LinearMulticlass/b [0.898180439959, -0.0512106496384, -0.19183247... "
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"m = LinearMulticlass(X, Y)\n",
"m.as_pandas_table()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Optimization terminated with:\n",
" Message: b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'\n",
" Objective function value: 0.000011\n",
" Number of iterations: 25\n",
" Number of functions evaluations: 26\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>class</th>\n",
" <th>prior</th>\n",
" <th>transform</th>\n",
" <th>trainable</th>\n",
" <th>shape</th>\n",
" <th>fixed_shape</th>\n",
" <th>value</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>LinearMulticlass/W</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>(none)</td>\n",
" <td>True</td>\n",
" <td>(2, 3)</td>\n",
" <td>True</td>\n",
" <td>[[8.19009259164, -17.4730464032, 6.94688807267...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>LinearMulticlass/b</th>\n",
" <td>Parameter</td>\n",
" <td>None</td>\n",
" <td>(none)</td>\n",
" <td>True</td>\n",
" <td>(3,)</td>\n",
" <td>True</td>\n",
" <td>[-5.00699554863, 7.85155019966, -2.18941734065]</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" class prior transform trainable shape fixed_shape \\\n",
"LinearMulticlass/W Parameter None (none) True (2, 3) True \n",
"LinearMulticlass/b Parameter None (none) True (3,) True \n",
"\n",
" value \n",
"LinearMulticlass/W [[8.19009259164, -17.4730464032, 6.94688807267... \n",
"LinearMulticlass/b [-5.00699554863, 7.85155019966, -2.18941734065] "
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"opt = gpflow.train.ScipyOptimizer()\n",
"opt.minimize(m)\n",
"m.as_pandas_table()"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"xx, yy = np.mgrid[-4:4:200j, -4:4:200j]\n",
"X_test = np.vstack([xx.flatten(), yy.flatten()]).T\n",
"f_test = np.dot(X_test, m.W.read_value()) + m.b.read_value()\n",
"p_test = np.exp(f_test)\n",
"p_test /= p_test.sum(1)[:,None]"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.PathCollection at 0x10a330fd0>"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAFpCAYAAAB0/VUQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xd8U/X+BvDnZDVNdyktm0IpU8reGxREkSVGQFBQAb3+\n1CtDBFmKqMj1iuuiskEEI4KMy1KGDFkiS2ZZBYEChe40SZOc3x8YboHSJm2Sk5M879eLFzT50PN4\nbMuTk+85RxBFEUREREREgUwhdQAiIiIiIqmxFBMRERFRwGMpJiIiIqKAx1JMRERERAGPpZiIiIiI\nAh5LMREREREFPJZiIiIiIgp4LMVEREREFPBYiomIiIgo4LEUExEREVHAU0m0Xd5bmoiIiIi8RShu\nQKpSjCtXrki1admKiYlBWlqa1DFkhfusZLjfSob7zXXcZyXD/VYy3G+u84d9VqFCBafmuHyCiIiI\niAIeSzERERERBTyWYiIiIiIKeCzFRERERBTwWIqJiIiIKOCxFBMRERFRwGMpJiIiIqKAx1JMRERE\nRAGPpZiIiIiIAh5LMREREREFPJZiIiIiIgp4KqkDEBERkXdcT0lD8u/nYcu3oWzlMqjVKgEKBY+P\nEQEsxURERH7v4vHLMLy/Bke3nbjr8bJVy+Dxlx9Gx2daQRAEidIR+QaWYiIiIj925sB5fDRwFsy5\n5vueu5FyEwve+h7Xzt9A/4m9JEhH5Dv4ngkREZGfspjy8dmLcwstxAWt/3oL/th41EupiHwTSzER\nEZGf2rf2IDJvZDs1u2nurx5OQ+TbWIqJiIj81N5Vfzg9e+K3ZGRcz/JgGiLfxlJMRETkp7Ju5rg0\nn+3iPJE/YSkmIiLyU9qQIJfmg8O0HkpC5PtYiomIiPxUw4frOT1bqVZ5lKkY5cE0RL6NpZiIiMhP\ntdO3gEardmq283Ntea1iCmgsxURERH4qNCoEQ6Y/Xexc/Y510GFAKy8kIvJdvHkHERGRH2vzZDOo\ng1RYMnkFMq7dfXUJpVqJdvoWGPTuk1CplRIlJPINLMVERER+rnmPRmjcLQkHNx1F8u/nYbXYULZK\nNFr1aYrI2HCp4xH5BJZiIiKiAKBSK9Hs8YZo9nhDqaMQ+SSuKSYiIiKigMdSTEREREQBj6WYiIiI\niAIeSzERERERBTyWYiIiIiIKeCzFRERERBTw3HZJNr1erwTwO4DLBoOhh7s+LxERERGRp7nzSPHr\nAE648fMREREREXmFW0qxXq+vBOBxAHPc8fmIiIiIiLzJXUeKZwJ4E4DdTZ+PiEgWkpOTpY5ARERu\nUOo1xXq9vgeA6waD4YBer+9YxNxwAMMBwGAwICYmprSbDjgqlYr7zUXcZyXD/eac1NRUdOzYEZcu\nXUJsbCz3Wwlwn5UM91vJcL+5LpD2mSCKYqk+gV6v/wDAYABWAFoA4QBWGAyGQUX8NfHKlSul2m4g\niomJQVpamtQxZIX7rGS435xXp04dDBkyBGPHjuV+KwHus5LhfisZ7jfX+cM+q1ChAgAIxc2VevmE\nwWAYZzAYKhkMhngA/QFsKaYQExH5jVatWmHt2rVSxyCiv4miCLudqznJdW67JBsRUSAaOnQoBg4c\nyH+EiSRkzbdh35qD2LJ4J84dTIHdJqJslTJoP6AlOg5shbDoUKkjkgy4tRQbDIZtALa583MSEfmy\ndu3aQaFQYPXq1XjxxReljkMUcHIzjPhk6DdI3n/+rsevp6Rh+YdrsWnONoxc9BKqJVWWKCHJBe9o\nR0RUSjVr1sT3338vdQyigCOKIj4fPu++QlxQVloO/jVoFm5dzfBiMpIjlmIiolLq3r07Dh06JHUM\nooBzfNdpnPit+Msi5tzKxc/ztnshEckZSzERUSk9++yzyMrKAq+qQ+Rd25bsdnp2+7I9sOZbPZiG\n5I6lmIiolGJiYhAeHo6vv/5a6ihEAeWvk86/EM1Jz0XGtUwPpiG5YykmInKDxo0b89JsRN4mFHvp\n2dLNU0BhKSYicgO9Xo9Tp05JHYMooFSpU8Hp2fCYUETFRXgwDckdSzERkRs88cQTsNls+PXXX6WO\nQhQwOg1q4/RshwGtoFQpPZiG5I6lmIjIDRQKBRISErBw4UKpoxAFjFotE5DUqU6xc5Gx4Xh4aHsv\nJCI5YykmInKTPn36YPdu58+GJ6LSEQQB/5g1BPXa1XzgTHT5SIz57mVExoZ7MRnJEUsxEZGbvPrq\nq8jKysLly5eljkIUMIJDtRj97ct4fe6LeKhDbejCg6EJ1qBK3YoY9O6TeH/LOFSq7fzaYwpcbr3N\nMxFRIIuNjUV0dDTmzJmDyZMnSx2HKGAolAo07lYfjbvVlzoKyRiPFBMRuVHz5s2xefNmqWMQEZGL\nWIqJiNxo4MCBOH/+POx2u9RRiIjIBSzFRERu1KlTJwiCgK1bt0odhYiIXMBSTETkRgqFAlWrVsWy\nZcukjkJERC5gKSYicrPOnTtj7969UscgIiIXsBQTEbnZ0KFDcfPmTWRmZkodhYiInMRSTETkZvHx\n8QgJCcHixYuljkJERE5iKSYi8oAmTZpg1apVUscgIiInsRQTEXnAoEGDcOrUKV6ajYhIJliKiYg8\noHv37gCAjRs3SpyEiIicwVJMROQBCoUCNWrUwNKlS6WOQkRETmApJiLykG7dumH//v1SxyAiIiew\nFBMRechzzz2HrKwspKWlSR2FiIiKwVJMROQh5cqVQ3h4OC/NRkQkAyzFREQe1LBhQ6xfv17qGERE\nVAyWYiIiD+rbty9Onz4tdQwiIiqGSuoARET+rE+fPnjjjTewf/9+NGvWTOo4Ps9qseLAhiPYt+Yg\nsm/lIjhMi4YP10OrPk2hDQmSOh4R+TGWYiIiD1KpVKhUqRLmzZvHUlyMswdT8MXwebh1NeOuxw/9\ncgyG99dg2CfPoHG3+hKlIyJ/x+UTREQe9sgjj2DXrl1Sx/Bpl05cwUcDvryvEDsYs/Lw+fB5OLL1\nhJeTEVGgYCkmIvKw4cOH4+bNm7w0WxGWvbcKphxzkTN2mx2LJvzAW2cTkUewFBMReVjlypURHh6O\nuXPnSh3FJ6Weu44/fz3p1OyNlJs4us25WSIiV7AUExF5QfPmzbFhwwapY/ik0/vOuTh/1kNJiCiQ\nsRQTEXlB//79cfbsWb71X4h8i9W1ebNr80REzmApJiLygm7dukEURezYsUPqKD4npmKUR+eJiJzB\nUkxE5AUKhQJVqlTBd999J3UUn1OvfW1ExoU7NatSK9GydxMPJyKiQMRSTETkJZ07d8aePXukjuFz\nVGolug3r5NRsu6dbIDwmzMOJiCgQsRQTEXnJkCFDkJaWhpycHKmj+JxHh3dE+/4ti5yp164mBk7p\n66VERBRoWIqJiLwkISEBwcHBWLJkidRRfI5CocDzM/pj2MxnULV+pbuei4uPwYDJvTFy4QhotGqJ\nEhKRv+NtnomIvCgpKQmrV6/GiBEjpI7icwRBQNt+zdHmyWa4nnIT2TdzoAvXolxCLBQKHsMhIs/i\nTxkiIi/S6/U4fvy41DF8miAIiIuPQY0m8aiQWI6FmIi8gj9piIi8qF+/frBarbw0GxGRj+HyCSIJ\nXM7JwdoLZ3EhKwsCgJpRUegRn4CY4GCpo5GHqVQqxMfHY/78+WjXrp3UcYiI6G8sxeRROfkWXM3N\nhUIQUDEkFFpVYH/JWWw2fHzwd6y7cA5igce3X/kL84//iacTa2FE/QZQCnwTx59169YNy5YtkzoG\nEQWAW1cysO273/DHxqPIyzYhJEqH5o83RPv+LXl5w3sEdkMhjzmXmYElp05gy18Xkf/3bW1DVGp0\nj6+GATVrI04XInFC77OLIibv/Q07rvxV6PM2UcR3p0/CaLVidONmXk5H3vTCCy9g1qxZSE9PR1QU\n785GRJ6xfdkeLBhngC3fduextL9uIeXoX1j16UaM+HQwmj7WQMKEvoWHo8jt9qZexbAtm7Dx4oU7\nhRgAcq35WH7mNF7YvBFnMtIlTCiNHVf+emAhLuinc2fw5800LyQKDFa7HSfTb+GP69dwNjMDoigW\n/5c8rHz58ggNDcXixYuljkJEfmrf2oOYO3rpXYW4IEtePr58eQGO7Tzl5WS+i6WY3OpKbg7e3r0D\nZlvh34QAkGE2Y8yuX2G05nsxmfRWnEl2enblWednqXBmmw2LThyDfv0avLh5I17bvgXP/bwegzet\nw09nk2GXuBw3bNgQ69atkzQDEfknu82OZVNXOTX3/XurfeJggS/g8glyqx/PnIapiELscCMvD5su\npqB39Rpu27ZdFHHgeip2XrmMnPx8RGiC0Oeh+qikVEEQBLdtpyRsoh0Hb1x3ev7A9WseTOP/8qxW\njNm5DYfSbtz33IXsLPzr4O84nHYDE5q3lGz9dp8+fTB+/HhJtk3uc/7IJWxb8hsun74KQRBQpW5F\ndBrUGpVqV5A6GgWwI1uP4+Zl596RTfnzL5w7dBEJjap6OJXvYykmtxFFEetTzjs9/9/z59xWik+m\n38LUfbuRkp111+OGM6dQOyoak5u3RuUw6U4osNrtsMP5V+Imm9WDafzfzEMHCi3EBf18KQXVwiPw\nbJ16Xkp1t759+2L06NE4fPgwGjTgmj65MeWa8fVri/HHxqN3PX563zn8smAHWj/ZDM9/1B/qIP4z\nS9537tBFF+dTWIrB5RPkRkarFVkWi9PzqcYct2z3dPotvPrr5vsKscPJ9Ft4ZdsvuJLrnu2VhEah\nRJha4/R82WCdB9P4t5umPGxw8sXZD2dOId9e/DsbnqDRaFC+fHksWLBAku1TydmsNnw2bO59hbig\n337cj69eXcS3pUkSdpu9+KECRBfn/RVLMbmNysW7TqkVylJvUxRFTP9jP/KsRR9ZvWU24fPDf5R6\neyUlCAK6VY13ev5RF2bpbr9cTIHNySKSbjZjb2qqhxM9WPv27bF9+3bJtk8l8/u6wzi2vfiTk35f\ndxh/OjFH5G7lqse6NB/n4ry/YikmtwlSKlE7Ktrp+QYxZUu9zRPpt3Aq/ZZTs7uuXEaqMbfU2yyp\nJxNqQu3EC4dQtRqPxyd4IZF/up5n9Oi8Ow0fPhypqanIyir8XQ7yTZsX7nR6dssi52eJ3KXZ4w0Q\nHKZ1aja6QiQeal/Lw4nkgaWY3KpPQqLTs71dmH2QvalXnZ61A/j9mnRHBSuHhWFy89ZQFXHSn1ap\nxAet2yMyKMiLyfyLRunaOxBBLs67U61atRASEoKFCxdKloFcY7fbkfy78+dOnN53zoNpqDiZN7KQ\neu46sm9Jt3xOCkG6IHQf0dmp2Z6vdYVSJd3PQV/CMwDIrbpWqYp1F87hcDEnOXWrEo+kMjGl3l6e\ni5d1MxazzMLTOlaqjC+CH8aik8ew++qVO6feKQUBHStWxnN16qF6RKSkGeWucdk4LD553KlZAUDD\nGGnfNmzSpAlWr16NV199VdIc5BzRLrq0XtNq4Umz3ma327Hnpz/wy4IdOPvHhTuP125VA0/+swcS\nW8dLfkUib3jitUdw62oGti357cEzrz6Cjs+09mIq38ZSTG6lVigxvU17TNn7G/Y84Chuj/jqGNW4\nqVt+KEVrg12aL6N17u0kT3qoTAw+atMB141GXMzOgiAA1cIjXP5vocI1iY1D5dAwXMrJLna2Rbny\nqBga6oVUDzZgwAAWYhlRqpSIKheB9NRMp+ZjKjm/pIxKz26z45t/fovdKw/c99zJ3WcwbfdMPDyk\nHQZNfdLvi7FCocCQD/Vo0KUuNi/ciT9/PQkAEBQCGj3yEB4e2g712nLZREEsxeR2oWoNZrTpgBPp\nt7D63BmkZGdBKQhIjIxC7+qJqBoe7rZtda5UBf85esipGzGEqNRoXb6i27ZdWrE6HWJ1vMqEuykE\nAWMaN8PIHVthLeLrIlStxv8lNfJissL16NEDr7zyCnbu3Im2bdtKHQcAYDaacfH4FeSb81GmQhTi\nqpV+/b8/afd0C6z+dJNzs/1beDgNFbTiX+sKLcQF/bJgB8pWLYNHh3XyUirpCIKAxl3ro3HX+jAb\nzcjLNkEXHgxNsPNXQwokLMXkEYIgoG50GdSNLuPR7cTqdOhcqQp+uZRS7GzP6gkIVvFLPhA0jo3D\nR206YOr+3Ug3m+97vmJIKN5r1Rbx4RESpLubQqFA5cqVsWTJEslLcVZaNlZ/tgk7f9iHvGzTnccT\nm1XDYy91QeNu9SVM5zs6DWqDn+duR16Oqci58JhQtHuKpdhb8nJM2DTPuau5rJu1BQ8PaQ+VOnDW\n0gbpghCk4/kqReGJdiR7oxo1Rc3IqCJnmsbGYVi9JC8lIl/QvFx5/PhYL0xq3gqdK1VB87hy6FYl\nHtNbt8d3jz6OxGK+ZrypQ4cO2LNnj6QZ0v66hXee+Dd+nrf9rkIMAMn7z+PTF+ZgzefOHR31d9Hl\nI/H6vBcRFPLgghESqcMbC4YjJJLvBnnL/rWHYM69/0VwYTKvZ+HIVufOPaDAwcNmJHthGg2+6NAF\nc48fxX8vnENO/v9OvosJ1uGJ+Op4tk7dIq+LLIoicvLzYRXtCFNrXL7mMvkmjVKJrlXi0bVKvNRR\nivTcc89h0aJFMBqN0EmwpMZut+PTF+Yg7VLRlzdcPv2/qFy3Ihp2keYugL6kTutETPnvKKyftRm7\nfzqAfPPtE+qCQoLQtl8zdB/RGWWrePadMrrb9Ys3XZtPSfNQEpIrlmLyCzq1Gq82aIxh9ZJwKO06\ncvPzEaEJQufadZCV/uD7v5usVqw5fxY//b32Gbi91vTRqtXwVI2aqBgq3a2hKXDUrl0bWq0WBoMB\nQ4YM8fr2T+xKxsVjl52aXf/VFpbiv1WoEYcXPh6IgVP64salmxAEAbFVy/AtaokoVa4dzOBlyOhe\nPluKsywWbEg5h11XriDXervgdKlcBV0qV0GQ0mdjk8S0KhValqtw5+OirlmbbjJh5M6tSM7IuOvx\nnPx8LD9zGmvPn8X7rdqhebnyHstL5FC3bl389NNPkpTincv3Oz17cvcZ3Lx8C2Uq8qoKDsFhWlSp\n6zsn8QaqGk2quTSf2NS1efJ/Pvke8eZLF9H3v6vw2eGDOHDjGk6m38Lea1fx/u970W/d6mKvgUtU\nHLsoYvzuHfcV4oJMNhvG796BFN5tjLygb9++OHr0qCTbvnXlwe+mFMbZy5EReVO9djURW9W5699X\nb1gV8fUrezgRyY3PleKdV/7ClL27YLIVfsHzdLMZo3ZsxWknb+1LVJgD11Nx9Gbx68lMNhuWJZ/0\nQiIKdAMGDIDZbMbBgwe9vu0gFy/PpNaqPZSEqOQUCgWeebcvBEXR1x9WqZUYMKmXl1KRnJS6FOv1\n+sp6vX6rXq8/rtfrj+n1+tdL+rlsoh2fHDqA4q44a7LZ8OXRQyXdDBHWnHf+1qs/X7wAk8R3wiP/\np9VqUbFiRcyZM8fr267d2vlbrofHhKJiYjkPpiEquYZd6uEf/3nugdfhDYnQ4fV5w1CzeYKXk5Ec\nuONIsRXAKIPBUBdASwCv6PX6uiX5RLuvXsU1o9Gp2QPXr+FiNt/WppK5lOP8147JZsONvDwPpiG6\nrWvXrtixY4fXt9v+6RZQBzl3rkaHga2h0vC8DvJdzXs0wif7pqD/xF6o2bw6KtUqj9ota2Dwe/2w\nIPlTJHWqI3VE8lGl/slmMBiuArj695+z9Xr9CQAVAbh8AcCjN11bK3z0ZhqqhLnv7mgUOBSCa68H\nFX5+O1DyDS+88ALmzZuHzMxMRER478YioVEhGDC5DxaN/6HIuQo1y6H7CP+/CxjJX2hUCLqP6Izu\nIzrf9bguXAdjmnMH3yjwuPXlvl6vjwfQCMDekvz9fJvdxXlbSTZDpXAhKxOrzp3BqfR02EURVcLD\n0LNaDdSLLiOr+8jXiYrGKSfXpUcFBSGOt2MmL4iPj0dISAi+/fZbvPLKK17ddpdn20IQgKXvroIl\nz3Lf87VaJOCVr4YgJILfC0TknwRRLG4Fr3P0en0ogF8BTDMYDCsKeX44gOEAYDAYmlgs9//QXXT0\nMN7f5dwtGgFgzuO90LZylRJnlhuVSgWrRGtb8202vLtjG344WfgbAK0rVcbMR7ojPMi3rs/5oH12\nMu0Gei9f5tTneKlxU/yzeSt3R/NpUn6tyZk79tsjjzyC3Nxc/Pbbb25K5ZrcTCM2L9mBo9tPwGLK\nR2zlMnj42Q6o2bS6R1748mutZLjfSob7zXX+sM80Gg0AFPsDzC2lWK/XqwGsBbDRYDD824m/Il65\ncuW+BzPMZvT570/Itxd/xDhOp4Oh+xNQuvg2uJzFxMQgLc37d+ARRRHTft+DDSkXipyrXyYGn7bv\nXOS1gb2tqH02bf8erE85X+Tfj9PpMKdLN0QFaT0Rz2dJ9bUmd+7Yb99++y0mTZqEc+ecPxlUzvi1\nVjLcbyXD/eY6f9hnFSpUAJwoxe64+oQAYC6AE04W4geKDApCz2rOnRE6sGadgCrEUjp262axhRi4\nvcZ748Xi55xxKv0W/nvhHNaeP4sTt27CXe9oFPRmk2boVsTtfyuFhuHT9p0DrhCTtPr16wez2Yzj\nx10+LYOIiErBHWuK2wAYDOCoXq93XCdtvMFgWFeST/Z/DRrhqjEXv129/0iyQ78aNdE3wflLCFHp\nrDyb7NLsE06+sCnM3tSrmH3sCE7es943MTISL9ZNQpsK7rtrlFqhxIRmLdE3IRErzyXjz5tpsNpF\nVAgJQY9qCehYsbJPHfWmwKDVahEXF4f58+djxowZUschIgoY7rj6xE44cUjaWWqFEu+3aoe1F85h\n5dnTOJv5vzsnNYgpi341aqJjxcqyOqlL7o64cAfB0xnpMFmt0Kpc/9LadPEC3tu3G4UtnknOyMBb\nv23H2CbN0aMUpftegiCgXpkY1Cvj3F2QiLyhbdu22L7d+fMriIio9HzyYpMqhQK9q9dAr2oJuGY0\nIiffgsigIMQE86xnKVicWON977yrCw4u5+Tg/f17Ci3EDiKAj/7Yj4fKxCA+3HuXqyLytqFDh2LF\nihWwWCyOE0SIiMjDfHpRriAIKBcSghqRUSzEEqoQEuL0bJhag1C167eA/elcMqxOrBu2i6JLyzmI\n5KhRo0bQaDRYsmSJ1FGIiAKGT5difySKIv68mYbZfx7Bp4cOYNGJY7icky11rCI9Fl/dhdlqJbrR\nxeZLFz0ySyRXDRs2xPLly6WOQUQUMHxy+YS/Ss5Ixwe/78XpjPS7Hv/m2BG0LV8RY5s298krHTxc\nuSrmH/8T1/OKvguQVqlE34SaJdpGhtnk/KzFDLso8i5z5Nf0ej3GjRsndQwiooDBI8VekpyRjle2\n/XJfIXbYefUy/m/bZmRZzF5OVrxglQr/atsB0UUU9iClEu+1aouKoaEl2oZO5fySi2CVioWY/F6/\nfv2Qn5+P/fv3Sx2FiCggsBR7gSiKeG//bhiLuSNMSnYWvjp62EupXFM9IhJzunSDvkatu9YMqxUK\nPFo1HrM7d0XLchVK/Plbl3f+77oySyRXKpUKlSpVwsKFC6WOQkQUELh8wgsOp92469JyRdl08QJe\nrt8QYT54xnmsTofXGjbGiPpJuJyTC7toR7mQEISqS5+1b0Ii1hVzd7n/zZZsiQaR3HTo0AE///yz\n1DGIiAICjxR7wa6rl52eNdls+OPGNQ+mKb0gpQrVIyJQIzLKLYUYAGpHl8Hg2nWLnXs6sRYaxJR1\nyzaJfN1zzz2Ha9euwWRyfs09ERGVDEuxF+Tm53t03l8Mr5eE/0tqVOgl3XQqFUY81AD/l9RIgmRE\n0qhbty6CgoJ4FQoiIi/g8gkviAgKcmk+0sV5fyEIAvrXrI1e1Wtgy18XcTYjAyJEVI+IRJfKVVw6\nGY/IX9SuXRsrV67EoEGDpI5CROTXWIq9oHOlKlh88rhTs+EaDZrElvNwIt8WrFLhcReujUzkz3r0\n6IFPPvlE6hhERH6Pyye8IDEyCo3Kxjo126taDQQplR5ORERyMWjQIBiNRiQn806ORESexFLsJROa\ntUScruhbVTcuG4uhdR/yUiIikoPw8HDExcXh66+/ljoKEZFfYyn2kjhdCL7u9AgerlwVyntuPBGq\nVmNgzdqY0bYjNDxKTET36NSpE7Zu3Sp1DCIiv8Y1xV4UE6zDlBat8WqDRvj9WiqMViuigrRoUa48\nglX8X0FEhRs+fDiWLVuGnJwchJbwrpG+6srZVGxYuAVZN7Kh0WmQ1KkOarVIgMC7VhKRl7GJSaCM\nNhjdqlaTOgYRyUStWrUQHByMJUuWYMSIEVLHcYusmzmYN2YpDm76867H//vlL6hUqzxe/PdAVGtQ\nRaJ0RBSIuHyCiEgGkpKSsHbtWqljuEVuhhEf9PvsvkLs8Nepq3i/32c4ezDFy8mIKJCxFBMRyUDv\n3r1x/Lhzl3b0dcs/WosryUXfudOSl4/ZbyyBKIpeSkVEgY6lmIhIBvr16weTyYRTp05JHaVU8rJN\n2LV8v1OzV89cw/Fdpz2ciIjoNpZiIiIZ0Ol0iI2NxcKFC6WOUion95yB2Whxev7Qz8c8mIbulXkj\nC9uW/Ia1X/yMzYt24uaVdKkjEXkNT7QjIpKJli1byv7SbMasPJfm83JMHkpCBeWk52LJ5BXYu+Yg\nbPm2O48vnrAcTbrVx6Cp/RBVLkLChESexyPFREQyMXjwYFy6dAlWq1XqKCUWHhPm0nxYtH9dgs4X\n5aTn4v1+n+G3Fb/fVYgBQLSL+H39EbzXZybSUzMlSkjkHSzFREQy0bp1a6hUKqxYsULqKCVWu2UN\nhMc4X3Rb9GzkwTQEAEvf/QmXT6UWOZN26Rbmv7nMS4mIpMFSTEQkI3Xq1MH3338vdYwSUwep0Hlw\nW6dma7ZIQHz9yh5OFNiybuZgz08HnJo9vOU4rl1I83AiIumwFBMRyUi/fv1w6NAhqWOUyhOvPoL6\nHWoXORNdIRIvfTbIS4kC1+HNx2C9Z8lEUQ6sP+zBNETSYikmIpKRZ555BmazGUePHpU6SompNCr8\nc/4wPPFa1/vWDCvVSrTs1RgTV72BMhWjJUoYOHIzjC7N57g4TyQnvPoEEZGMaLValC9fHvPmzcMn\nn3widZxVj9gsAAAgAElEQVQSU2lU6Pfm4xj6zgD8umIXMm9kQxuiQZ02NREZGy51vIChCw92aT4k\nQuehJETSYykmIpKZ9u3b49dff5U6hlsEBWvQuFt9t3wuq8WKnAwjgoI1CA7TuuVz+rukznWhVCvv\nu+rEgzTu9pCHExFJh6WYiEhmnn32WXz//fewWCzQaDRSx5HcxeOXsXHONuxd9QfyzbcvVxefVBld\nnm2L1k82g0qtlDih74qMDUfzHg2xe2XxJ9s91KE2yifEeSEVkTS4ppiISGYaNGgAtVot60uzucvO\n5fswufu/sNOw704hBoALRy5h7uil+HjwVzAbzRIm9H0Dp/RFueqxRc5ElYvA8x897aVERNJgKSYi\nkqFatWoFfCk+uecM5oz8Dnab/YEzx3eexpxRS72YSn7Cy4Ti7ZWvo+ljDSAohPueT+pUBxNX/ZMn\nPpLf4/IJIiIZeuyxx/Dll19KHUNSaz//GaJdLHZu35qD6DOqOyrU4Fv/DxJeJhSvfvM8bl5Jx4H1\nR5CTngtdRDAaPfwQ4qqVlToekVfwSDERkQwNHjwYOTk5uHTpktRRJHHz8i0c/fWk0/Pbl+7xYBr/\nUaZCFLq+0AF9Rz+GR4d1YiGmgMJSTEQkQ1FRUShTpgzmzJkjdRRJpJ6/4eL8dQ8lISJ/wVJMRCRT\nbdq0waZNm6SOIQml0rUrSigU/OeOiIrGnxJERDI1bNgwXLp0CSaTSeooXlepdnmoNM6fFlO9YRUP\npiEif8BSTEQkU40bN4ZGo8GyZcukjuJ1oVEhaNGzkVOzSrUS7fu39HAiIpI7lmIiIhmrV68eVq5c\nKXUMSfR6vRt0EcXfprjHKw8jPCbMC4mISM5YiomIZKxXr144duyY1DEkEVetLN787h+IKPvgwvvY\ny13QZ1R3L6YiIrnidYqJiGSsf//+mDx5Ms6ePYuEhASp43hdtQZV8NHOidi98gB2/bgP6Vczodaq\nUbdNTXR+tg0q1SovdUQikgmWYiIiGQsNDUVMTAzmz5+P9957T+o4ktCGBKHToNboNKi11FGISMa4\nfIKISOZatmyJbdu2SR2DiEjWWIqJiGRu4MCBSElJgd1ulzoKEZFssRQTEclcu3btoFAosG7dOqmj\nEBHJFksxEZHMKRQKJCYmYunSpVJHISKSLZZiIiI/0LNnT+zfv1/qGEREssVSTETkB4YMGYLc3Fwk\nJydLHYWISJZYiomI/EB4eDhiY2MxZ84cqaMQEckSSzERkZ9o164dL81GRFRCLMVERH5iyJAhuHz5\nMqxWq9RRiIhkh6WYiMhPNG7cGGq1GitXrpQ6ChGR7LAUExH5kcTERKxYsULqGEREssNSTETkR7p1\n64ZDhw5JHYOISHZYiomI/Mhzzz2HrKwspKamSh2FiEhWWIqJiPxITEwMIiMjMX/+fKmjEBHJCksx\nEZGfad68OTZs2CB1DCIiWWEpJiLyM0OGDMG5c+dgs9mkjkJEJBssxUREfqZ9+/ZQKpW8CgURkQtY\niomI/IwgCKhTpw6WLVsmdRQiItlgKSYi8kM9e/bEwYMHeXc7IiInsRQTEfmhRx55BADw888/S5yE\niEgeWIqJiPxQdHQ01Go13n33XRiNRqnjEBH5PJZiIiI/dOPGDURGRqJdu3bQ6/XYu3cvRFGUOhYR\nkc9SueOT6PX6RwF8CkAJYI7BYPjQHZ+XiIhKZsGCBejevTsmTZqEJUuWYNSoUQgODkbHjh2RkJCA\ncuXKoXz58oiMjERoaCiCg4OhUPA4CREFrlKXYr1erwTwJYBHAPwFYL9er19tMBiOl/ZzExGRa0RR\nxBdffIHdu3djxYoVyMjIQNmyZTFgwAD8/vvvWLduHbKysmAymZCfnw+bzQZRFCGKIgRBuOuXQqG4\n75dSqbzvl0qlgkqlglqthlKphFqthlqthkajgUajgUqlgkajQVBQEDQaDbRaLYKCghAUFITo6GjY\n7XZotVpotVoEBwff+bNOp4NOp0NwcDCCg4PvfMzyTkSe4I4jxc0BnDEYDOcAQK/XLwPQCwBLMRGR\nF928eRNjx47F8ePH0aZNG+j1evz1119o1qwZatSogfbt22PgwIGIjIy8UzbVajUAwG63w2QywWQy\nIS8vD3l5eTCbzcjLy4PJZILZbL7ze8FfFosFFosFZrMZVqv1zsf5+fnIy8tDZmYmrFYrrFbrnRLu\n+NhRyPPz82G32+/75SjrRS37uLfEFyzzgiBAqVTeVeZVKtV9f1ar1XdKvaPAF/yz45ej1DsKvePP\njhJfsNgXLPGOP2u1WhZ6Ih/mjlJcEcClAh//BaCFGz4v/c1is2Hb5UvYtHsHLqSnQykIeKhMDHpX\nr4GHysRAEASpIxKRhGw2G+bPn48ZM2ZArVYjLCwM5cuXx4ABA5CUlASVyi0r5TwiJiYGaWlpTs/b\n7XYYjcY75d1oNMJoNN4p7QXLvOMxR2m/t8jn5+cjPz//TqF3fJydnY38/Pw75b1gib/3170lvuCR\n96IKvStH5R3l3XFEXqVSQavVQhCEEhd5x6+CR+YdZb7gEfvg4GBoNBqWeQoIXvtJqdfrhwMYDgAG\ngwExMTHe2rSsXczMxIhf1uN8RsZdj1/OzcHGixfwWI1EfNjpEWiUSokS+jaVSsWvtRLgfisZb+83\nURSxcOFCjBs3DpmZmejcuTPefPNNtGvXTjYvlgPpa81qtcJoNCI7O/tOmXf82Ww2Iycn574j9Y6P\nCxZ7k8kEq9V65+P8/HyYTCZkZWUVekT+3kJvt9sLLfQlKfMFC/29R+ULK/SOpTUFl9gEBQVBrVbf\nddS94DKbggXdceS9YIkPCQm56/fQ0FBoNJpC8wfS15u7BNI+c0cpvgygcoGPK/392F0MBsM3AL75\n+0PRlSMDgSrTbMaLmzfiqjH3gTPrziTDarFgUvPWXkwmH64ehaLbuN9Kxlv7TRRFLFmyBNOnT0dG\nRga6deuGd999FxUqVABwexmFXATi15rjKG1UVFSJP4c395vFYrlT1O89Sn/vEXrHEhyLxXJnuU3B\nJTaOI/P5+fmwWCwwGo3IyMiAxWK5b5mNo9g7SvyDirzdbgcAl4u84/eCZb6wUl/YennHL5VKdVep\nd/x+79H5gkfk7y34jmU2vrpm3h++Rx0/G4vjjlK8H0CiXq+vhttluD+AgW74vAFv+ZnTRRZih00X\nU6BPrI3aUdFeSEVEUhFFEQsWLMDHH3+MrKws9OjRAx988AEiIiKkjkZ+zLEUQ05fZxaLBbm5uXeV\neLPZDI1Gg2vXrt0p9AXXyhc8Gu84+u4o747S7vjYcWTfsczm3rXyzh6ZL6rQO97tKVjgHevkHb8X\ndtKr4/cHFXnHEfrCivy9y2uCgoLw9NNP+1xR95RSl2KDwWDV6/X/B2Ajbl+SbZ7BYDhW6mQBziba\nsfr8WafnV507g9pNmnswERFJRRRFzJ49GzNnzkROTg769OmDadOmITQ0VOpoRD7JUfDuPRrv60c9\nHWvmHUtpjEYjcnNz71pWU/AIveOovKOoO353FHrHya+OI/Q5OTl3yn3BEl/YEXnH+vjExEQ0bdpU\n6l3jFW5ZU2wwGNYBWOeOz0W33cjLw01TntPzJ27J5+1SInKO3W7HrFmz8Nlnn8FkMqFfv36YOnUq\ndDqd1NGIyAMUCgVCQ0N96gWvr7+QcCffPSU5wNldvPOUq/NE5LvsdjtmzpyJr776ChaLBQMGDMCU\nKVMQFBQkdTQiIr/FUuyjymiDEaxSIc9qdWq+cliYhxMRkafZ7XbMmDEDs2fPhs1mw6BBgzBp0qQ7\n1xImIiLPYSn2UUFKJbpVicdP5844Nf9EtQQPJyIiT7FarXj//fexcOFCAMDQoUPx1ltv+fT1hYmI\n/A1/4vqwpxNrY+PFC8UeLa4bXQbN48p7KRURuYvFYsHUqVPx7bffQqlUYvjw4RgzZkzAnOlNRORL\n+JPXh1UOC8OHrdshuIijRQkRkfiwdTsoZHKhfiICTCYTxo0bh5o1a+L777/Ha6+9htOnT2Ps2LEs\nxEREEuGRYh/XJLYcFj7SHT+eOY31KReQZTEDAKqGhaN39RroUS2hyNJMRL7DaDRi4sSJWL58OYKD\ngzFmzBi8/PLLLMJERD6AbUoGKoSE4tUGjTGp08M4d/UKVIICoWq1bG7jShTocnJyMH78ePz0008I\nCQnBhAkTMGzYMKljERFRASzFMqJUKBAVpJU6BhE5KTMzE2PHjsW6desQHh6O9957D88++6zUsYiI\nqBAsxUREbpaeno7Ro0dj06ZNiIqKwvTp0zFgwACpYxERURFYiomI3OT69et4/vnnsXnzZsTExGDm\nzJl48sknpY5FREROYCkmIiql69evY9SoUdi2bRtiY2PxxRdfoFevXlLHIiIiF7AUExGV0NWrVzFy\n5Ejs2LED5cuXx7Jly9CmTRupYxERUQnwOkBERC66dOkSnnrqKTRr1gznz5/HggULsH//fh4dJiKS\nMR4pJiJy0oULFzBy5Ejs27cPVapUwZIlS9ChQwepYxERkRuwFBMRFePcuXN44403cODAAVSrVg3L\nli1D27ZtpY5FRERuxFJMRPQAycnJGDlyJA4ePIiEhAT8+OOPaNGihdSxiIjIA1iKiYjucfLkSYwc\nORJHjhxBYmIiVq1ahSZNmkgdi4iIPIilmIjob3/++SdGjRqFY8eOoXbt2lizZg0aNWokdSwiIvIC\nlmIiCniHDx/G6NGjceLECdStWxfr169H/fr1pY5FRERexFJMRAHrwIEDGD16NJKTk1G/fn1s2rQJ\ndevWlToWERFJgKWYiALOnj17MHbsWJw9exaNGjXC1q1bkZiYKHUsIiKSEEsxEQWMHTt2YNy4cbhw\n4QKaNm2K7du3o3r16lLHIiIiH8BSTER+b/PmzXj77bfx119/oXnz5liyZAmqVq0qdSwiIvIhLMVE\n5Lc2bNiASZMm4cqVK2jbti1+/PFHVKxYUepYRETkg1iKicjvrF69Gu+88w6uXbuGjh07YvXq1ShX\nrpzUsYiIyIexFBOR3/jxxx/x3nvvIS0tDZ07d8bGjRsRExMjdSwiIpIBlmIikr2lS5figw8+QHp6\nOh599FF89NFHiIqKkjoWERHJCEsxEcnWokWL8NFHHyErKwuPPfYYpk+fjoiICKljERGRDLEUE5Gs\n2O12zJ07F5988glycnLQu3dvvP/++wgNDZU6GhERyRhLMRHJgt1ux6xZs/D5558jLy8P/fr1w9Sp\nU6HT6aSORkREfoClmIh8mt1ux6effopZs2bBYrFgwIABmDx5MrRardTRiMinWRGE3VALpwGIsIrx\ngNhH6lDkw1iKicgn2e12zJgxA7Nnz4bdbsczzzyDiRMnQqPRSB2NiHxcMNYjVJgLpZD2vwcFQMz4\nAjoMhBFP3X6AqACWYiLyKVarFR9++CHmz58PURQxZMgQjB8/HioVf1wRUfF0+AHhii8LfU4QMxCu\n+A8U4i3kiC95ORn5Ov4rQ0Q+wWKxYOrUqViyZAkUCgVGjBiB0aNHQ6FQSB2NiGRCiRSECf8pckYU\ngVBhGcxiK+SjgZeSkRywFBORpEwmE9555x0sW7YMKpUKr776Kl5//XWWYSJymU5YDUEQi5wRBMfs\nSmSKLMX0PyzFRCQJk8mECRMm4IcffoBWq8XIkSPxyiuvsAyTF5mhwhUAImwoBxG8koncabHThdld\nyIQIri0mB5ZiIvKqnJwcTJgwAStXroROp8P48eMxbNgwlmHyGgWuIUT4HsHYCIWQCwAQxSDkoQty\nxf6woYrECamkBOQ4PyvkA6IFQJDnAgUUM5RIBQDYEAdAflcIYikmIq/IysrCuHHjsGbNGoSFhWHK\nlCkYOnSo1LEowKiQjGhhNBRC5l2PC4IZOqyDFluRIb4PCxpJlJBKw44IKJDr1KwoBgHg1WxK6/aL\nTAOCseHOi0y7GAwTHkGu+DRsqChxQufx0AwReVRmZiZeeukl1KtXD9u3b8f777+PY8eOsRCT1wnI\nQZQw9r5CXJBCyEOk8DYUuO7FZL5EBGD++3f5MaGDC7MdwaUTpaPCKcQIwxEi/HinEAO3v490wmqU\nEYZDjaMSJnQNjxQTkUekp6dj9OjR2LRpE6Kjo/Hxxx9Dr9dLHYsCWDA2QincKnZOIRihw2rkiC96\nIZVvUOMIdMJKaLETgpAPUdTAhPYwin2Qj3pSx3NantgTIfgBgmB94Iwo3j7ZLpc38igVAdmIEt4q\n5kVmLqIwHmniAthRxovpSoZHionIrdLS0jBkyBAkJSXh4MGD+PTTT3H48GEWYpJcsLDe+Vk4Pytv\nIkKFOSijeA3Bwtbb62wBCIIFwcIvKKN4BSFYLHFG59lQHpniaIjig48ACwKQbR8OK2p7MZn/CcZ6\nKIX0YucUQjaCsdYLiUqPR4qJyC2uX7+OUaNGYevWrYiLi8OXX36Jnj17Sh2L6A4lrjo/K9z8+yQs\n/15zqsNKhArf3jl6ei9RBMIUc2G3RyMPj3s/YAmY8ChEMRyh+AZq4cJdz4mKSsi0PgsTukoTzo+4\n8iJTJ6xDrvicB9O4B0sxEZXK1atXMWrUKGzfvh3ly5fH7Nmz0b17d6ljEd1HhAZw+iQsAf7/T2Q+\nQoTbR4ELK8QFHw8VFiJPfBSA0jvRSsmM1jCLraAWj0KNMwBEWFEV4VFdYbpZ/BIaKp5rLzKvAaIN\nvv714+/f8UTkIZcvX8bIkSOxa9cuVKxYEfPmzUPXrjz6Qr4rH/WhxHYnZ+vB31cYBmGXU29/A4BS\nuI4gcR/MaOXhVO4kIB9JyEdSgYf8+/+pdzlfIUVRCTl8P/l+QiLyKZcuXUK/fv3QokULXLx4EYsW\nLcLevXtZiMnnGcXby3nEIi6s4HjOKPbyQiJpqZDi0rzSxXnybxbUd3o2Hw9BDlf6YCkmIqecO3cO\nvXv3RqtWrZCamoqlS5di9+7d6Ny5s9TRiJxiQROYxA4QhMKLsWNdrVlsBBM6eT+g17laUny/1JD3\nOF44+tOLTJZiIipScnIynnjiCbRv3x7p6ekwGAzYuXMn2rVrJ3U0IhcJyBDfRp74SKFraAUBMIkt\nkSFOQyCsLrQi0aPz5N8saA6T2NqJF5mNYUJ77wcsAf//rieiEjl58iRGjhyJI0eOIDExEStXrkSz\nZs2kjkVUShpkim8jVxyAYGH1XSdhGcWeAXWZLjOawybG3T4JqhhWsTIsaOiFVCQfCmSIkxGBDxAs\nbLvv2dsvMlshU5wIudRNeaQkIq85evQoxo4diyNHjqB27dpYs2YNGjXiLW/Jv1hRHdniP6WOITEl\nssVhiBTeK3YyWxwGvrlM9wtCpjgFueJp6IQ1UOEMAMCKasgTeyAfdSCnZTcsxUQEAPjjjz8wZswY\nnDp1Cg0aNMD69etRv77zJ1IQkfyY8DCy7FkIE76AINjve14UFcgSR8Isk7e/SRpW1ESWOErqGKXG\nUkwU4Pbu3YuxY8fizJkzSEpKwi+//IK2bdsiLS1N6mhE5AVG9IVZbAYdViMIO6BANuwIhwkdkCf2\nhA0VpI5I5BUsxUQBaufOnRg3bhzOnz+PRo0a4ddff0VCQoLUsYhIAjZURrb4CrLxitRRiCTDUkwU\nYLZs2YIJEybg4sWLaNGiBRYvXoz4+HipYxEREUmKpZgoQGzatAkTJ07E5cuX0aZNG3z//feoXLmy\n1LGIiIh8AksxkZ9bu3YtJk+ejGvXrqFDhw5YtWoVypUrJ3UsIiIin8JSTOSnVqxYgalTpyItLQ2d\nO3fGxo0bERMTI3UsIiIin8RSTORnli5dig8++ADp6eno2rUrZsyYgejoaKljERER+TSWYiI/sWjR\nInz00UfIysrCY489hunTpyMiIkLqWERERLLAUkwkc3PmzMG///1v5OTkoFevXvjggw8QGhoqdSwi\nIiJZYSkmkiG73Y6vvvoKn332GfLy8tCvXz9MnToVOp1O6mhERESyxFJMJCN2ux2fffYZ/vOf/8Bi\nsaB///6YMmUKtFqt1NGIiIhkjaWYSAbsdjv+9a9/4ZtvvoHNZsOgQYMwceJEaDQaqaMRERH5BZZi\nIh9mtVoxffp0zJs3D6IoYsiQIRg/fjxUKn7rEhERuRP/ZSXyQVarFdOmTcPChQshCAJeeOEFvPnm\nmyzDRFQIEWqcgArJAETYUBUWNACgkDoYkayU6l9YvV4/A8ATACwAzgIYajAYMtwRjCgQWSwWvPvu\nu1iyZAlUKhX+8Y9/YOTIkVAo+I8bEd1Pg70IE2ZDLZy563GrWBk54hCY0EWiZETyU9p/aX8G8JDB\nYEgCcBrAuNJHIgo8JpMJb731FmrWrAmDwYA33ngDp06dwujRo1mIiahQWvyCKGEc1MIZiOLdz6mE\nS4hUTIUOBmnCEclQqY4UGwyGTQU+3AOgX+niEAUWo9GIiRMnYvny5QgODsaYMWPw8ssvswgTUdFs\nVxEhfAhBsAMABKHwsXDFf2CxN4AVtbwYjkie3LlA8XkA37vx8xH5rZycHEyYMAErV66ETqfDhAkT\nMGzYMKljEZFMKMwGCILVqVmdsBJZ4lseTkQkf4J473su99Dr9b8AKFfIU28bDIZVf8+8DaApgL4G\ng6HQT6jX64cDGA4ABoOhicViKU3ugKRSqWC1OvdDkG7ztX2WlZWFV199FT/88AMiIiIwefJkvPTS\nS1LHuo+v7Te54H5zXYn2mWiFkL8dguVnQMwChHCImi4Q1R0BITBORlVlPg7YUpyaFREMW9S+Bx9O\nDiD8HnWdP+yzvy9fWuw3QLGluDh6vX4IgBEAuhgMBqOTf028cuVKqbYbiGJiYpCWliZ1DFnxlX2W\nlZWFN998E+vWrUN4eDjGjh2LwYMHSx3rgXxlv8kN95vrXN1nKpxEpDAFKiH1vudsYhwyxEnIRz13\nRvRJccqeEMQsp+dT7RsA8CY//B51nT/sswoVKgBOlOLSXn3iUQBvAujgQiEmChjp6ekYO3Ys1q9f\nj6ioKEyfPh0DBgyQOhaRLKlwBtHCG1AIeYU+rxSuIQojcUv8zP/X0Aqht4+SO0EUVQB4ox+i4pT2\nbJ4vAIQB+Fmv1x/S6/VfuSETkezdunULQ4cORVJSEvbv34+ZM2fiyJEjLMREpRAufPrAQuygEMwI\nF2Z6KZF0RHVHp2fNaA1es5ioeKW9+kQNdwUh8gfXr1/H6NGjsWXLFsTGxuLzzz9H7969pY5FJHsq\nnIVGOApRLHpprCgCGuEEVOJpWFHTewG9zK59GoJpKQThwUsgHfvKKPbxYjIi+eJLRyI3SE1NxcCB\nA9G4cWOcOHECX3/9Nf744w8WYiI30eAPAMWfK+Z43jHvt5QJyBZHFDkiCECu+CQsaOSlUETyFhin\n6RJ5yKVLlzBq1Cj89ttvqFixIhYsWICHH35Y6lhEfkeA2aPzcmREf4j2UIQKc6EU0u96zi6GIFcc\niFwMlCgdkfywFBOVwIULFzBy5Ejs27cPVapUwbfffouOHTtKHYvIb9lQ1qV5u4vzcpWHHsgTu0Er\n7oRKSAYgwipWhQkdwatNELmGpZjIBWfPnsXIkSNx4MABxMfHY9myZWjbtq3UsYj8nhltYReDiz3R\nDgDsohYmtPNCKl+hhgmdALGT1EGIZI1riomccPLkSTz++OPo0KEDMjIy8OOPP2Lnzp0sxEReIiIE\neXji9p8fcG6Z4/E8PA4RYV5KRkT+gkeKiYrw559/YtSoUTh27Bhq1aqF1atXo3HjxlLHIgpI2eIw\nqJCCIGFvoc8LAmAWmxV7AhoRUWFYiokKcfDgQYwZMwYnT55EvXr1sH79etSvX1/qWEQBTo10cRp0\n4nKECCuhFK7decYmxsEo9kIungKgli4iEckWSzFRAXv37sXYsWNx5swZJCUlYdOmTahbt67UsYjo\nDhWM6A+j+BRUYjIUyIYdYbAiEYBS6nBEJGMsxUQAdu7ciXHjxuH8+fNo3Lgxtm7disTERKljEdED\nKWFFbalDEJEfYSmmgLZlyxZMmDABFy9eRPPmzbF48WLEx8dLHYuIiIi8jKWYAtKmTZswceJEXL58\nGa1bt8YPP/yAihUrSh2LiIiIJMJSTAFl7dq1mDJlClJTU9G+fXusWrUK5cqVkzoWERERSYylmALC\nihUrMHXqVKSlpaFz585Yt24dYmNjpY5FREREPoKlmPzawoULMX78eNy6dQtdu3bFjBkzEB0dLXUs\nIiIi8jEsxeSXFi9ejOnTpyMrKwuPPfYYpk+fjoiICKljERERkY9iKSa/MnfuXHz88cfIyclBr169\nMHv2bJhMJqljERERkY9jKSbZs9vt+Oabb/Dpp5/CaDSib9++mDZtGnQ6HUJDQ1mKiYiIqFgsxSRb\ndrsdX375JT7//HNYLBY89dRTmDp1KrRardTRiIiISGZYikl27HY7Zs6ciVmzZsFqteKZZ57BpEmT\noNFopI5GREREMsVSTLJht9sxY8YMzJ49G3a7HYMHD8bEiROhUvHLmIiIiEqHbYJ8ntVqxfTp0zFv\n3jyIoojnn38eb731FsswERERuQ1bBfksq9WKadOmYeHChVAoFBg+fDjGjBkDhUIhdTQiIiLyMyzF\n5HMsFgumTp2Kb7/9FiqVCi+//DJGjRrFMkwBxAwF0gGoYEc0AH7tExF5Gksx+QyTyYR33nkHS5cu\nhUajweuvv47XXnuNZZgChhIpCBGWQ4ufoRBuX0rQJsbBKPaEEb0gIlTihERE/oulmCRnMpkwceJE\nGAwGaLVajB49Gv/4xz9YhimgaLAHUcJkCIL5rseVwjWECbMRLG7CLfFfsKOsRAnJVwnIQDA2QC2c\nhwgBVjEBeegGEeFSRyOSFZZikozRaMTbb7+NFStWQKfTYdy4cRg+fDjLMAUcJVIKLcQFqYQUROFt\n3BRnAVB6Lxz5MBtChTkIwXIIQv7/HhaAMHE2cjEAOeJz4PIbIuewFJPX5eTkYNy4cVi1ahVCQ0Mx\naeGeTgcAAA3iSURBVNIkvPDCC1LHIpJMiGAoshA7qIXTCBL3wYxWXkhFPk0UES58DJ2wDqJ4/9OC\nYEEoFkJANrLF17yfj0iGWIrJazIzMzF27FisW7cOERERmDZtGgYPHix1LCKJmRGMX5yeDhb+C7PI\nUhzoBOueO4VYEAqfEUUgRFgBk9gF+ajn3YBEMsRSTB6Xnp6ON998Exs2bEB0dDSmT5+OAQMGSB2L\nyCcocdOpo8T/m7/swTQkF4Jp6e3fH1CICz6nE1YiU3RvKf7fOuZTAETYUAV5YnfYUN6t2yHyJpZi\n8pi0tDSMGjUKW7ZsQUxMDGbOnIknn3xS6lhEPkV0eX0wf2yTCCF/l9PTQdjvxm3bESosRAi+u3sd\nM4AQLEYeuiNL/CcAjRu3SeQd/OlKbpeamorRo0dj27ZtiIuLw5dffomePXtKHYvIJ9kRA5tYFkrh\nhlPz+ajr4UTk++wQ4Py7CwLy3LblMOErhAiGBzwrQiesgwKZyBDfBU8IJbnhKankNpcvX8bTTz+N\npk2b4vTp05g9ezYOHDjAQkxUJCWM4hPFTjlOpjKK/H4iJUQh2ulpG8q4ZasqJCNEMBR6Yh9we7mG\nKAJaYRe02O6WbRJ5E0sxlVpKSgqefPJJtGjRAikpKVi4cCH27duH7t27Sx2NSBaM6AOrWOmBzztO\npjKKj8OKBC8mI18lah53etaEh92yTZ2wCoCz65h/css2ibyJpZhK7OzZs+jVqxfatGmDa9euYenS\npdizZw+6dOkidTQiWRERhlvix8gXqxf6vKMQZ4lveDkZ+Sq7dgBEsfgVkKIY5NQ7Ec7Q4IBTc6II\naITDACxu2S6Rt3BNMbns5MmTGD16NA4dOoSEhAQsX74cLVu2lDoWkazZEYeb4jcIEvcgWPgvVLgC\nESrkoy6MYi9YUUPqiORLlFWQKY5DBKZBEOx3XZrN8WdRVCFDnAg7Yt2ySQEm5+YEx7wFIk+4Ixlh\nKSan/fnnnxg1ahSOHTuGWrVqYdWqVWjSpInUsYj8iApmtIVZbCt1EJIBE7rALkYjBAsRJBy687gg\nAP/f3v3HVlXecRx/31KhYGHiQIbaDebINiKK0zl/BJWBDMQNEfedQzZwRiOZ0cXyY9J00nQI6GSI\nSgKiGQ6GfIMTorWZP8F0kQVUFp3K2IioKFFXBLGUcnvP/rgXJUppe2n73NP7eSWEntunnE8eGvrh\nnOee50B0NvuiaznI6W12vhR96MLulo2NuhPRo83OLdIRVIqlWa+88grTp0/nzTffZPDgwVRXVzNk\nyJDQsURE8l4DZ9EQnUWX6G0KeQtIkGQgjTS9Rj1b+6NLOS6xrUVj0+uYtUJT4kWlWJq0adMmZsyY\nwbZt2xgyZAhPPfUUgwfrcVAiIrmmka/TyNfb9Rz7GU1x9GcKEp8cdVwUdaEuurJds4i0B/03Tr6k\npqaGYcOGMX78eIqLi3n++eeprq5WIRYRyWMRvdgd/Z5U1D19fNij2Q59HEUF7Il+S5KBARKKHBuV\nYvnM+vXrueCCC7j66qvp27cvNTU1PP744wwaNCh0NBERyQEHOZPa6H7qo4s5vEIcWsdcGy2gnkvD\nBRQ5Blo+ITzzzDOUlZWxc+dOzj//fFavXk1JSUnoWCIikoOSfJOPowoK+B+F0XYgRSMlNHJy6Ggi\nx0SlOI9VVVUxe/Zs3n//fS666CLWrl1L//79Q8cSEZEYSPFVGtpotzyRXKBSnIfWrVtHRUUFH374\nIcOHD6eqqoqTTmqb51iKiIiIxJFKcR5xd+bMmUNtbS0jR47k7rvv5sQTTwwdS0RERCQ4leI8sHLl\nSubPn8/u3bu57LLLmDdvHr179w4dS0RERCRnqBR3Yg8//DB33nkne/fuZezYscyfP59evXqFjiUi\n0o4aSFBHRHegW+gwIhIjKsWd0LJly1iwYAH79u3jiiuu4I477qC4uDh0LBGRdtOVzfRIPEY3XiSR\nSBFFBRzgXOqiK2ng+0AidEQRyXEqxZ1EKpVi6dKlLFy4kP3793PVVVdRWVlJjx7ae15EOrOI4sQS\nihOPpI8+21AiRVFiI0WJjXwaTeCT6CZUjEXkaFSKYy6VSrF48WLuvfde6uvrMTMqKyspKioKHU1E\npN314DGKE48QRekNJBKZ3nvo9yiC4xOP0hidRB0/CxdURHKeSnFMpVIpFi1axOLFi2loaGDixInM\nnj2brl27ho4mItJBkhyfWAF8XoK/6NDrxYm/UBeNB/RvpIgcmUpxzKRSKRYsWMCSJUtIJpNMmjSJ\n8vJylWERyTvdeJEuidoWjS1I7KEo+jv1DG/nVCISVyrFMZFMJikvL2fRokVEUcTkyZMpKyujsFB/\nhSKSnwp5p1Xju7RyvIjkFzWqHJdMJpkzZw7Lly+noKCA6667jhkzZqgMi0jeiyho5Ve0dryI5BM1\nqxzV0NBAZWUlK1asoLCwkKlTpzJ37lxqa1t2q1BEpLM7yHfadbyI5BeV4hxTX19PRUUFq1atomvX\nrtx8883ccsstFBQUUFCgqxwiIocc5EyS0TcoTOxodmwyOoUGvtcBqUQkrlSKc0RdXR3l5eWsWbOG\noqIipk+fztSpU1WERUSalOCT6EZOYBaJRNTkqChKj9PyCRE5GpXiwPbt20dZWRlr166lR48ezJo1\ni+uvv15lWESkBQ5wPnuiMr7CPBKJ5Jc+H0WF7ImmcYBhAdKJSJyoFAeyd+9eZs6cSVVVFT179qSi\nooIpU6aEjiUiEjv1jKQhGkr3qIqixAYK2EuKnhzgIuqisaQ4KXREEYkBleIOtnv3bmbOnEl1dTW9\ne/dm7ty5XHPNNaFjiYjEWoo+fMpkPo0mh44iIjGlUtxBPvroI6ZNm8azzz5Lnz59WLhwIRMmTAgd\nS0RERERQKW53u3btYtq0aaxfv55+/fpx3333MW7cuNCxREREROQwKsXtZOfOnZSWllJTU8PJJ5/M\nAw88wJgxY0LHEhEREZEjUCluYzt27KC0tJSNGzdSUlLC8uXLGTFiROhYIiIiInIUbVKKzawU+APQ\n190/aos/M262b9/OrbfeyubNmxkwYACrVq1i2DA9AkhEREQkDo65FJtZCTAKePvY48TP1q1bKS0t\nZcuWLZx22mmsWbOG8847L3QsEREREWmFttgh4o/ADKDp7YQ6oddff53Ro0czYsQI6urqWLduHRs2\nbFAhFhEREYmhYyrFZjYO2Onu/2yjPLGwYcMGRo0aRWNjI1VVVTz33HOcffbZoWOJiIiISJYSUXT0\nC7xm9gzwtSN8qgyYBYxy9z1m9hZwTlNris3sBuAGAHdXgxQRERGRjpJodkBzpbgpZjYEeBaoy7x0\nKvAecK6772rmaze7+zlZnTiPad5aT3OWHc1bdjRvrac5y47mLTuat9bLpznL+o127v4qfL6hfHNX\nikVEREREclVbvNFORERERCTW2mzzDncf0IrhS9vqvHlG89Z6mrPsaN6yo3lrPc1ZdjRv2dG8tV7e\nzFnWa4pFRERERDoLLZ8QERERkbzXZssnsqUtolvOzCqBcUAK+ACY4u7vhU2V+8zsLuDHQAPwX+Ba\nd/84bKrcZ2Y/BWYD3yX9VJnNYRPlLjMbDdwDdAGWufu8wJFynpk9BFwOfODup4fOEweZHWQfBvqR\n3jBrqbvfEzZV7jOzIuAFoBvp3rPG3W8PmyoezKwLsJn0nhSXh87T3oJeKc73LaKzcJe7n+HuQ4En\ngN+FDhQTTwOnu/sZwL+B2wLniYvXgCtJ/zCRJmR+aNwPjAEGAz83s8FhU8XCn4DRoUPETBIodffB\nwHnAr/W91iIHgB+6+5nAUGC0mWn72Za5BXgjdIiOEnr5RF5uEZ0td9972OHxaN5axN2fcvdk5nAj\n6WdqSzPc/Q133xo6RwycC/zH3be7ewPwCOk7OnIU7v4CUBs6R5y4+/vu/nLm409Il5VTwqbKfe4e\nufu+zOFxmV/6+dkMMzsVGAssC52lowRbPnH4FtFmFipG7JjZHOCXwB5geOA4cfQrYHXoENKpnAK8\nc9jxu8APAmWRPGFmA4CzgH8EjhILmTs6LwHfAu53d81b8xaSvnDZM3SQjtKupbglW0S35/nj6Ghz\n5u7r3L0MKDOz24CbAK2Lovl5y4wpI337cWVHZstlLZk3EcktZlYMPAr85gt3EKUJ7t4IDDWzE4DH\nzOx0d38tdK5cZWaH1vu/ZGaXhM7TUdq1FLv7yCO9ntkieiBw6CrxqcDLZtbsFtGdXVNzdgQrgSdR\nKQaanzczm0L6TT0j3F23zTJa8f0mTdsJlBx2fGrmNZE2Z2bHkS7EK939r6HzxI27f2xmz5Nez65S\n3LQLgZ+Y2WVAEdDLzFa4+6TAudpVkOUT2iI6O2Y2yN23ZQ7HAW+GzBMXmScDzAAudve60Hmk09kE\nDDKzgaTL8NXAxLCRpDMyswTwIPCGuy8InScuzKwvcDBTiLsDlwLzA8fKae5+G5k3pWeuFE/r7IUY\ncuCRbNIq88zs26QfybYDuDFwnri4j/SjeJ7O3JnY6O6au2aY2XjgXqAvUGVmW9z9R4Fj5Rx3T5rZ\nTcDfSD+S7SF3/1fgWDnPzFYBlwB9zOxd4HZ3fzBsqpx3IfAL4FUz25J5bZa7PxkwUxz0B5Zn1hUX\nAO7uTwTOJDlIO9qJiIiISN4L/Ug2EREREZHgVIpFREREJO+pFIuIiIhI3lMpFhEREZG8p1IsIiIi\nInlPpVhERERE8p5KsYiIiIjkPZViEREREcl7/wciIasFkZ94sgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11b2999b0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"for i in range(3):\n",
" plt.contour(xx, yy, p_test[:,i].reshape(200,200), [0.5], colors='k', linewidths=1)\n",
"plt.scatter(X[:,0], X[:,1], 100, np.argmax(Y, 1), lw=2, cmap=plt.cm.viridis)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"That concludes the new model example and this notebook. You might want to convince yourself that the `LinearMulticlass` model and its parameters have all the functionality demonstrated above. You could also add some priors and run Hamiltonian Monte Carlo using HMC optimizer `gpflow.train.HMC` and `sample` method. See the sparse_MCMC notebook for details of running the sampler."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
},
"widgets": {
"state": {},
"version": "1.1.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Computing file changes ...
