{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 104: Time Series with Pandas / NumPy\n",
    "\n",
    "Contributed by: Avi Thaker https://github.com/athaker/CNTK\n",
    "November 20, 2016\n",
    "\n",
    "This tutorial will introduce the use of the Cognitive Toolkit for time series data. We show how to prepare time series data for deep learning algorithms. We will cover training a neural network and evaluating the neural network model. We will also look at the predictive potential on classification of an Exchange-traded Funds ([ETF](https://en.wikipedia.org/wiki/Exchange-traded_fund)), and in this simplified setting how one could trade it. This tutorial serves **only** as an example of how to use neural networks for time series analysis. \n",
    "    \n",
    "It is important to note that the stock market is extremely noisy and is difficult to predict. This is best done by professionals with domain expertise. It is more important to make sure the model is correct before setting up a trading system (there are many factors to consider including but not limited to: [curve fitting bias](https://en.wikipedia.org/wiki/Overfitting), [forward looking bias](http://www.investopedia.com/terms/l/lookaheadbias.asp?lgl=no-infinite), profitability etc.). The learnings and anecdotes presented in this tutorial is only for exemplary purposes with the goal of introducing an approach to analyze time series data.\n",
    "\n",
    "This tutorial introduces how to use pandas_datareader package and pandas. Please note, this tutorial will utilize the numpy interface to CNTK which interfaces well with [Pandas dataframes](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) (a structure that is well suited towards timeseries analysis). \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import os\n",
    "import numpy as np\n",
    "import cntk\n",
    "import cntk.ops as C\n",
    "\n",
    "from cntk.blocks import default_options, Input   # Building blocks\n",
    "from cntk.initializer import glorot_uniform\n",
    "from cntk.layers import Dense                    # Layers\n",
    "from cntk.learner import sgd, learning_rate_schedule, UnitType\n",
    "from cntk.utils import get_train_eval_criterion, get_train_loss, ProgressPrinter, get_train_loss\n",
    "\n",
    "import datetime\n",
    "import pandas as pd\n",
    "pd.options.mode.chained_assignment = None  # default='warn'\n",
    "%matplotlib inline\n",
    "\n",
    "# Select the right target device when this notebook is being tested:\n",
    "if 'TEST_DEVICE' in os.environ:\n",
    "    if os.environ['TEST_DEVICE'] == 'cpu':\n",
    "        cntk.device.set_default_device(cntk.device.cpu())\n",
    "    else:\n",
    "        cntk.device.set_default_device(cntk.device.gpu(0))\n",
    "\n",
    "# If you want to set the program to use a CUDA enabled GPU\n",
    "#from cntk.device import set_default_device, gpu\n",
    "#set_default_device(gpu(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing stock data\n",
    "We first retrieve stock data using the method `get_stock_data`. This method downloads stock data on a daily timescale from Yahoo finance (can be modified to get data from Google Finance and many other sources). [Pandas datareader]( http://pandas-datareader.readthedocs.io/en/latest/remote_data.html) shows many use cases for this data reader."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# A method which obtains stock data from Yahoo finance\n",
    "# Requires that you have an internet connection to retreive stock data from Yahoo finance\n",
    "try:\n",
    "    from pandas_datareader import data\n",
    "except ImportError:\n",
    "    !pip install pandas_datareader\n",
    "    from pandas_datareader import data\n",
    "\n",
    "def get_stock_data(contract, s_year, s_month, s_day, e_year, e_month, e_day):\n",
    "    \"\"\"\n",
    "    Args:\n",
    "        contract (str): the name of the stock/etf\n",
    "        s_year (int): start year for data\n",
    "        s_month (int): start month\n",
    "        s_day (int): start day\n",
    "        e_year (int): end year\n",
    "        e_month (int): end month\n",
    "        e_day (int): end day\n",
    "    Returns:\n",
    "        Pandas Dataframe: Daily OHLCV bars\n",
    "    \"\"\"\n",
    "    start = datetime.datetime(s_year, s_month, s_day)\n",
    "    end = datetime.datetime(e_year, e_month, e_day)\n",
    "    bars = data.get_data_yahoo(contract, start, end)\n",
    "    return bars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Please check if there is trade data for the chosen stock symbol during this period\n",
    "data = get_stock_data(\"SPY\", 2000, 1,2,2017,1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building the training paramaters\n",
    "\n",
    "The stock market behavior exhibits substantial [autocorrelation](https://en.wikipedia.org/wiki/Autocorrelation) ([reference](http://epchan.blogspot.com/2016/04/mean-reversion-momentum-and-volatility.html)). We use [ETF](http://www.investopedia.com/terms/e/etf.asp) `SPY` index representing the \"market\" of stock. This is the ETF that encompasses around top 500 companies in America by market capitalization. We will trade under the assumption that there is some short term autocorrelation that have predictive power in the market. \n",
    "\n",
    "### Predicting\n",
    "* Whether or not the next data for the given stock/ETF will be above or below the current day.\n",
    "\n",
    "### Predictors\n",
    "* The previous 8 days, classified if greater than the current day,\n",
    "\n",
    "* The volume changes as a percentage,\n",
    "\n",
    "* The percentage change from the previous day.\n",
    "\n",
    "Note, we are not feeding the neural network the price itself. Financial timeseries data are noisy. It is important not to overfit the data. There is a lot we can do here (smoothing, adding more features, etc.), but we will keep this tutorial simple, and demonstrate CNTK's ability to interface with timeseries data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>Adj Close</th>\n",
       "      <th>diff</th>\n",
       "      <th>v_diff</th>\n",
       "      <th>p_1</th>\n",
       "      <th>p_2</th>\n",
       "      <th>p_3</th>\n",
       "      <th>p_4</th>\n",
       "      <th>p_5</th>\n",
       "      <th>p_6</th>\n",
       "      <th>p_7</th>\n",
       "      <th>p_8</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000-01-03</th>\n",
       "      <td>148.250000</td>\n",
       "      <td>148.250000</td>\n",
       "      <td>143.875000</td>\n",
       "      <td>145.437500</td>\n",
       "      <td>8164300</td>\n",
       "      <td>105.825332</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-04</th>\n",
       "      <td>143.531204</td>\n",
       "      <td>144.062500</td>\n",
       "      <td>139.640594</td>\n",
       "      <td>139.750000</td>\n",
       "      <td>8089800</td>\n",
       "      <td>101.686912</td>\n",
       "      <td>0.040698</td>\n",
       "      <td>0.009209</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-05</th>\n",
       "      <td>139.937500</td>\n",
       "      <td>141.531204</td>\n",
       "      <td>137.250000</td>\n",
       "      <td>140.000000</td>\n",
       "      <td>12177900</td>\n",
       "      <td>101.868820</td>\n",
       "      <td>0.001786</td>\n",
       "      <td>0.335698</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-06</th>\n",
       "      <td>139.625000</td>\n",
       "      <td>141.500000</td>\n",
       "      <td>137.750000</td>\n",
       "      <td>137.750000</td>\n",
       "      <td>6227200</td>\n",
       "      <td>100.231643</td>\n",
       "      <td>0.016334</td>\n",
       "      <td>0.955598</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-07</th>\n",
       "      <td>140.312500</td>\n",
       "      <td>145.750000</td>\n",
       "      <td>140.062500</td>\n",
       "      <td>145.750000</td>\n",
       "      <td>8066500</td>\n",
       "      <td>106.052718</td>\n",
       "      <td>0.054889</td>\n",
       "      <td>0.228017</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-10</th>\n",
       "      <td>146.250000</td>\n",
       "      <td>146.906204</td>\n",
       "      <td>145.031204</td>\n",
       "      <td>146.250000</td>\n",
       "      <td>5741700</td>\n",
       "      <td>106.416535</td>\n",
       "      <td>0.003419</td>\n",
       "      <td>0.404898</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-11</th>\n",
       "      <td>145.812500</td>\n",
       "      <td>146.093704</td>\n",
       "      <td>143.500000</td>\n",
       "      <td>144.500000</td>\n",
       "      <td>7503700</td>\n",
       "      <td>105.143175</td>\n",
       "      <td>0.012111</td>\n",
       "      <td>0.234817</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-12</th>\n",
       "      <td>144.593704</td>\n",
       "      <td>144.593704</td>\n",
       "      <td>142.875000</td>\n",
       "      <td>143.062500</td>\n",
       "      <td>6907700</td>\n",
       "      <td>104.097201</td>\n",
       "      <td>0.010048</td>\n",
       "      <td>0.086281</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-13</th>\n",
       "      <td>144.468704</td>\n",
       "      <td>145.750000</td>\n",
       "      <td>143.281204</td>\n",
       "      <td>145.000000</td>\n",
       "      <td>5158300</td>\n",
       "      <td>105.506992</td>\n",
       "      <td>0.013362</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000-01-14</th>\n",
       "      <td>146.531204</td>\n",
       "      <td>147.468704</td>\n",
       "      <td>145.968704</td>\n",
       "      <td>146.968704</td>\n",
       "      <td>7437300</td>\n",
       "      <td>106.939489</td>\n",
       "      <td>0.013395</td>\n",
       "      <td>0.306428</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Open        High         Low       Close    Volume  \\\n",
       "Date                                                                   \n",
       "2000-01-03  148.250000  148.250000  143.875000  145.437500   8164300   \n",
       "2000-01-04  143.531204  144.062500  139.640594  139.750000   8089800   \n",
       "2000-01-05  139.937500  141.531204  137.250000  140.000000  12177900   \n",
       "2000-01-06  139.625000  141.500000  137.750000  137.750000   6227200   \n",
       "2000-01-07  140.312500  145.750000  140.062500  145.750000   8066500   \n",
       "2000-01-10  146.250000  146.906204  145.031204  146.250000   5741700   \n",
       "2000-01-11  145.812500  146.093704  143.500000  144.500000   7503700   \n",
       "2000-01-12  144.593704  144.593704  142.875000  143.062500   6907700   \n",
       "2000-01-13  144.468704  145.750000  143.281204  145.000000   5158300   \n",
       "2000-01-14  146.531204  147.468704  145.968704  146.968704   7437300   \n",
       "\n",
       "             Adj Close      diff    v_diff  p_1  p_2  p_3  p_4  p_5  p_6  p_7  \\\n",
       "Date                                                                            \n",
       "2000-01-03  105.825332  0.000000  0.000000    0    0    0    0    0    0    0   \n",
       "2000-01-04  101.686912  0.040698  0.009209    0    0    0    0    0    0    0   \n",
       "2000-01-05  101.868820  0.001786  0.335698    1    0    0    0    0    0    0   \n",
       "2000-01-06  100.231643  0.016334  0.955598    0    0    0    0    0    0    0   \n",
       "2000-01-07  106.052718  0.054889  0.228017    1    1    1    1    0    0    0   \n",
       "2000-01-10  106.416535  0.003419  0.404898    1    1    1    1    1    0    0   \n",
       "2000-01-11  105.143175  0.012111  0.234817    0    0    1    1    1    0    0   \n",
       "2000-01-12  104.097201  0.010048  0.086281    0    0    0    1    1    1    0   \n",
       "2000-01-13  105.506992  0.013362  0.339143    1    1    0    0    1    1    1   \n",
       "2000-01-14  106.939489  0.013395  0.306428    1    1    1    1    1    1    1   \n",
       "\n",
       "            p_8  \n",
       "Date             \n",
       "2000-01-03    0  \n",
       "2000-01-04    0  \n",
       "2000-01-05    0  \n",
       "2000-01-06    0  \n",
       "2000-01-07    0  \n",
       "2000-01-10    0  \n",
       "2000-01-11    0  \n",
       "2000-01-12    0  \n",
       "2000-01-13    0  \n",
       "2000-01-14    1  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Feature name list\n",
    "predictor_names = []\n",
    "\n",
    "# Compute price difference as a feature\n",
    "data[\"diff\"] = np.abs((data[\"Close\"] - data[\"Close\"].shift(1)) / data[\"Close\"]).fillna(0) \n",
    "predictor_names.append(\"diff\")\n",
    "\n",
    "# Compute the volume difference as a feature\n",
    "data[\"v_diff\"] = np.abs((data[\"Volume\"] - data[\"Volume\"].shift(1)) / data[\"Volume\"]).fillna(0) \n",
    "predictor_names.append(\"v_diff\")\n",
    "\n",
    "# Compute the stock being up (1) or down (0) over different day offsets compared to current dat closing price\n",
    "num_days_back = 8\n",
    "\n",
    "for i in range(1,num_days_back+1):\n",
    "    data[\"p_\" + str(i)] = np.where(data[\"Close\"] > data[\"Close\"].shift(i), 1, 0) # i: number of look back days\n",
    "    predictor_names.append(\"p_\" + str(i))\n",
    "    \n",
    "\n",
    "    \n",
    "# If you want to save the file to your local drive\n",
    "#data.to_csv(\"PATH_TO_SAVE.csv\")\n",
    "data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### What we are trying to predict\n",
    "\n",
    "Here we are trying to predict whether or not the next days' trading will be above or below the current day. We will represent a predicted up day as a 1, else a 0 if the next day is the same or below. (Note: the market is unlikely going to close at the same price as it did the previous day). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data[\"next_day\"] = np.where(data[\"Close\"].shift(-1) > data[\"Close\"], 1, 0)\n",
    "data[\"next_day_opposite\"] = np.where(data[\"next_day\"]==1,0,1) # The label must be one-hot encoded\n",
    "\n",
    "# Establish the start and end date of our training timeseries (picked 2000 days before the market crash)\n",
    "training_data = data[\"2000-01-15\":\"2008-01-01\"]\n",
    "\n",
    "# We define our test data as: data[\"2008-01-02\":]\n",
    "# This example allows to to include data up to current date\n",
    "\n",
    "test_data = data[\"2008-01-02\":\"2016-11-25\"] \n",
    "training_features = np.asarray(training_data[predictor_names], dtype = \"float32\")\n",
    "training_labels = np.asarray(training_data[[\"next_day\",\"next_day_opposite\"]], dtype=\"float32\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Here we are actually building the neural network itself. We will use a simple feedforward neural network (represented as `NN` in the plots) with 10 inputs and 50 dimensions.\n",
    "\n",
    "**Suggested exercise** Try experimenting here with different sizes and number of hidden layers. What happens when we make this network large? Do we overfit the data? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2000, 10)\n"
     ]
    }
   ],
   "source": [
    "print(training_features.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, we are using `layers` library in this tutorial. The details documentation can be found [here](https://cntk.ai/pythondocs/layerref.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Lets build the network\n",
    "input_dim = 2 + num_days_back\n",
    "num_output_classes = 2 #Remember we need to have 2 since we are trying to classify if the market goes up or down 1 hot encoded\n",
    "num_hidden_layers = 2\n",
    "hidden_layers_dim = 2 + num_days_back\n",
    "input = Input(input_dim)\n",
    "label = Input(num_output_classes)\n",
    "\n",
    "def create_model(input, num_output_classes):\n",
    "    h = input\n",
    "    for i in range(0,num_hidden_layers):\n",
    "        h = Dense(hidden_layers_dim, \n",
    "                  init = glorot_uniform(), \n",
    "                  activation = C.relu)(h)\n",
    "        \n",
    "    return (Dense(num_output_classes, init=glorot_uniform(), activation=None)(h))\n",
    "    \n",
    "z = create_model(input, num_output_classes)\n",
    "loss = C.cross_entropy_with_softmax(z, label)\n",
    "label_error = C.classification_error(z, label)\n",
    "lr_per_minibatch = learning_rate_schedule(0.125,UnitType.minibatch)\n",
    "trainer = cntk.Trainer(z, loss, label_error, [sgd(z.parameters, lr=lr_per_minibatch)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Initialize the parameters for the trainer, we will train in large minibatches in sequential order\n",
    "minibatch_size = 100\n",
    "num_minibatches = len(training_data.index) // minibatch_size\n",
    "\n",
    "#Run the trainer on and perform model training\n",
    "training_progress_output_freq = 1\n",
    "\n",
    "# Visualize the loss over minibatch\n",
    "plotdata = {\"batchsize\":[], \"loss\":[], \"error\":[]}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Number of passes through the data, how we train time series data\n",
    "This tutorial will differ from other tutorials in the sense that here we will not randomly send data to the trainer, instead each minibatch will be fed sequentially in the order of the time dimension. This is key to time series data-handling where we want to \"weigh\" the data at the end of our sample a slightly higher. You can put in multiple passes, however you will notice significant performance degradation. Try it out! Additionally, multiple passes tend to overfit the financial timeseries data. This overfitting can be mitigated using standard ML approaches such as [L1 regularization](https://en.wikipedia.org/wiki/Regularization_(mathematics)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of mini batches\n",
      "20\n",
      "The shape of the training feature minibatch\n",
      "(100, 10)\n"
     ]
    }
   ],
   "source": [
    "tf = np.split(training_features,num_minibatches)\n",
    "\n",
    "print(\"Number of mini batches\")\n",
    "print(len(tf))\n",
    "\n",
    "print(\"The shape of the training feature minibatch\")\n",
    "print(tf[0].shape)\n",
    "\n",
    "tl = np.split(training_labels, num_minibatches)\n",
    "\n",
    "# It is key that we make only one pass through the data linearly in time\n",
    "num_passes = 1 \n",
    "\n",
    "# Defines a utility that prints the training progress\n",
    "def print_training_progress(trainer, mb, frequency, verbose=1):\n",
    "    training_loss = \"NA\"\n",
    "    eval_error = \"NA\"\n",
    "    if mb%frequency == 0:\n",
    "        training_loss = get_train_loss(trainer)\n",
    "        eval_error = get_train_eval_criterion(trainer)\n",
    "        if verbose: \n",
    "            print (\"Minibatch: {0}, Loss: {1:.4f}, Error: {2:.2f}%\".format(mb, training_loss, eval_error*100))\n",
    "    return mb, training_loss, eval_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minibatch: 0, Loss: 0.6971, Error: 49.00%\n",
      "Minibatch: 1, Loss: 0.6826, Error: 50.00%\n",
      "Minibatch: 2, Loss: 0.7160, Error: 54.00%\n",
      "Minibatch: 3, Loss: 0.7018, Error: 49.00%\n",
      "Minibatch: 4, Loss: 0.7088, Error: 52.00%\n",
      "Minibatch: 5, Loss: 0.6905, Error: 42.00%\n",
      "Minibatch: 6, Loss: 0.7083, Error: 54.00%\n",
      "Minibatch: 7, Loss: 0.7015, Error: 51.00%\n",
      "Minibatch: 8, Loss: 0.7068, Error: 57.00%\n",
      "Minibatch: 9, Loss: 0.7128, Error: 60.00%\n",
      "Minibatch: 10, Loss: 0.6881, Error: 46.00%\n",
      "Minibatch: 11, Loss: 0.6859, Error: 40.00%\n",
      "Minibatch: 12, Loss: 0.7035, Error: 52.00%\n",
      "Minibatch: 13, Loss: 0.6803, Error: 40.00%\n",
      "Minibatch: 14, Loss: 0.6927, Error: 51.00%\n",
      "Minibatch: 15, Loss: 0.7098, Error: 55.00%\n",
      "Minibatch: 16, Loss: 0.6880, Error: 48.00%\n",
      "Minibatch: 17, Loss: 0.6885, Error: 45.00%\n",
      "Minibatch: 18, Loss: 0.6895, Error: 45.00%\n",
      "Minibatch: 19, Loss: 0.7105, Error: 51.00%\n"
     ]
    }
   ],
   "source": [
    "# Train our neural network\n",
    "tf = np.split(training_features,num_minibatches)\n",
    "tl = np.split(training_labels, num_minibatches)\n",
    "\n",
    "for i in range(num_minibatches*num_passes): # multiply by the \n",
    "    features = np.ascontiguousarray(tf[i%num_minibatches])\n",
    "    labels = np.ascontiguousarray(tl[i%num_minibatches])\n",
    "    \n",
    "    # Specify the mapping of input variables in the model to actual minibatch data to be trained with\n",
    "    trainer.train_minibatch({input : features, label : labels})\n",
    "    batchsize, loss, error = print_training_progress(trainer, i, training_progress_output_freq, verbose=1)\n",
    "    if not (loss == \"NA\" or error ==\"NA\"):\n",
    "        plotdata[\"batchsize\"].append(batchsize)\n",
    "        plotdata[\"loss\"].append(loss)\n",
    "        plotdata[\"error\"].append(error)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAACfCAYAAAA8qTSuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm4FMXV/z9fQHFFcUMJStyNGFReFVxQXHBJBA0GI4ZE\n4hJj9Of2atQYA4J5I2pI1GgMcUPiEoLirqCCElBcQAQEIYobIIuIiKJs9/z+qB4ZLnfuna2nZ+ae\nz/PMw3R3ddXpZm6frqpzviUzw3Ecx3EKoUnSBjiO4ziVjzsTx3Ecp2DcmTiO4zgF487EcRzHKRh3\nJo7jOE7BuDNxHMdxCsadiZM1kv4m6epcy0o6QtLH8Vr3bbvvSzqqFG1VA9H/zdRil83Djv9I+nkc\ndTuloVnSBjjJI+kDYHugtZl9lrb/TWBf4Ltm9pGZnZdtnXWUzSuhSVJb4H2gmZnV5FNHtSDpMOAZ\nwr1sAmwCfAko2re3mc3JpU4zewn4frHLOo0P75k4EB5E7wO9Ujsk7QNsTJ5OoIikHpSKvSGpadxt\nFIKZjTOzzc2sBdCOcF+2SO2r7UgUkYixTqPDnYmTYihwRtr2GcCQ9AKS7pHUP/p+hKSPJV0qaYGk\nuZL61FV27S5dJWmRpNmSTk878ANJkyQtlfShpL5p570U/fu5pC8kdYzOOUfS9GjfNEn7pZ2zv6S3\nJC2R9KCkDeu6YElnSBonaZCkRUBfSX0lDU0r01ZSjaQm0fYYSf2j876Q9KykrTLUP13SD9K2m0pa\nKGk/Sc0lDZX0aWTnq5K2raueBljHWUTDRf0lvUzotewo6ay0e/VfSWellT9a0vtp2x9LukTSlMiu\n+yVtkGvZ6PhVkj6Jyp0d3cedGrygwO8lfSBpvqS7JW0eHds4aid13yak7n90ne9H1/mupFPzuJ9O\nnrgzcVJMADaXtGf04PwJ8E/q7xFsD2wOtAbOBm6TtEU9ZbeKyvYBBkvaPTr2JfAzM9sC+CHwK0nd\no2OHR/+2iN6+X5XUE/g90Dt6S+8OLE5rqydwLLAzYZiuTz3X0BF4F2gF/CHaV7s3Vnu7F8HZbgs0\nBy7LUPcDwOlp28cDi8xscnR+C+A7hPvyK+DreuzMhd6Ea24BzAXmAydE9+oc4Nao55mi9vX1BI4G\ndgEOAH6Wa1lJJwLnA0cAewBH1XFuJs4h3LfDgV0J9+cv0bFfEHrMraP9vwa+iZzNn4Cjo+s8FJiS\nZXtOEXBn4qST6p10BWYA8xoovxIYYGZrzOwZglPYM0NZA64xs1VmNhZ4CjgVwMzGmtnb0fdpwEOE\nh1A66U7tLOAGM5sUnTPbzNIn+G82swVm9jnwBJDea6nNXDO73cxqzGxFA9eb4h4zey8qP6ye+h8E\nukvaKNruFe0DWAVsDexhgTfN7Mss22+Iu81sVvT/ssbMnjKzDwHM7EXgBaBzPef/2cwWmdkS4Enq\nv3+ZyvYE7ors+Bq4Ngf7TwduiubpvgJ+y1qnvArYhrX3bZKZLY+O1QDfl9Q8+v9/J4c2nQJxZ+Kk\n80/CH20f4L4syi+uNSm+HNgsQ9klZvZN2vaHhLdLJHWUNDoaAvocOJfwwMjEjsB79RxfkKVNAPlE\nmc3Ppn4zew+YDnSTtDGhB/VAdHgoMBJ4SNIcSdereHM261yTpBOj4aDFkpYQXhbqu7+53L9MZVvX\nsuNjsp/3ak34faT4EGgeDQPeCzwPDIuGz/5PUhMzW0Zw1hcA8yU9ntbzdUqAOxPnW8zsI8JE/AnA\nI0WuvmX0QE2xE2t7PvcDjwLfMbMtgb+z9sFT19DIx4Thj2JQu/6vCFFSKXYosP6HCA76JOBtM5sN\nYGarzWyAmbUDDgG6AcUKjf32mqJe0b8JQ3jbmllL4DniD2j4BGiTtr0T2Q9zzQPapm23BVZEPaBV\nZtbfzPYGDgN6AD8FMLORZtaVMKT6HuF35JQIdyZObc4EjoqGJoqJgGslbSCpM2FuZFh0bDNCz2WV\npINYd55hEWH4It153AlcJqkDgKRdJe1YJDsnA4dL2jGa/7mywPoeIszfnMfaXgmSukjaJ5qf+pIw\nfJNr6HM2DqE5sAHwKWDRXMbRObaTD8OAsyTtIWkT4Hc5nPsgcGkU/LA5cB3RvZN0pKR2kkTafZO0\nfdQD2xhYTXgpWFPMC3Lqx52JA2lvjGb2fmouovaxXOqpg0+AJYS3zqHAuWb23+jYr4EBkpYSHjr/\nSrPna8Jb9XhJn0k6yMyGR/sekPQFMIIwGZurvetfgNnzUftTgNcJcy7rFMmxvvnAK0An0q6L8PY8\nHFgKvA2MIdyXVMLn7dlU39A+M1sKXELo+S0mvMnXvqaG6sy5rJk9CfwNGAvMBMZFhzLNS6XX9Q/C\nvfoPIThiKXBxdKw1ode8FJgKjCI4mqbA5YTf1yLgYEIAgFMiFPfiWJKOJ0RiNCFMyA2sdfwyQjfV\nCG9Q3yOM525GGLdvRXhj+4eZ3RKd05LwY2sLfACcGv3ROI5ThkTRYxPNrHnStjjxEKszibrwswjd\n6nmEN73TMkVZRF3wi83sGEnbA9ub2WRJmwETgZPM7B1JAwmTvzdIugJoaWaFDkc4jlNEJJ1MiNrb\nnPBi+JWZ/SRZq5y4iHuY6yDgv2b2oZmtIowfn1RP+W9DJ81sfhSPTxQyOYMQk09URyqhbghwcgy2\nO45TGOcT5mpmESK9LkjWHCdO4tbm+g7rhgfOITiY9Ygmzo6njnFOSd8lxK9PiHZtZ2YLIDgdSdsV\nz2THcYpBFFnlNBLKSeixGzAuSjT7lmiIazhwUZTAVBd1jtVJSlpXynEcpyIxs5zCx+Me5ppLiC9P\n0SbaVxensTY7GABJzQiOZKiZPZZ2aIGkVlGZ7YGFmQwwM/8U6dO3b9/EbaiWj99Lv5/l/MmHuJ3J\n68BuUbz4hgSH8XjtQlE8/xHAY7UO3Q1MN7Oba+1/nLV6S2fUcZ7jOI5TQmJ1Jma2hjDpNooQS/+Q\nmc2QdK6kX6YVPRkYaWmJcpIOJYQMHyXpTQVV2eOjwwOBrpJmEiLFro/zOhzHcZz6iX3OxMyepZb4\nn5n9vdb2EGrJnZvZeEIiUl11fgYcU1xLnYbo0qVL0iZUDX4vi4vfz+SJPWkxSSRZNV+f4zhOHEjC\nymwC3nEcx2kEuDNxHMdxCsadieM4jlMwsTsTScdLekfSrEhHq/bxy9KitaZKWi1py+jYXQrri0+p\ndU7faEGhSbWivBzHcZwEKFuhx2j7MMKaBfeZWfu0cn2BZWY2qIH2fQLecRwnR8pxAj5voUcAMxtH\nWAOjLuJeKa5gJk6EW29N2grHcZz4iduZ1CX0+J26CqYJPT6cZd0XSJos6c4og77sGD0a3n03aSsc\nx3Hip+yFHjNwO9DfzEzSdcAg4Ky6Cvbr1+/b7126dClpctOkSXDCCSVrznEcJy9efPFFXnzxxYLq\niHvOpBPQz8yOj7avBMxqrbYYHXsEGGZmD9Xa3xZ4In3OJNvjSc+Z7LEHPPII7LNPYiY4juPkTDnO\nmRQq9AhhbkS1ym+fttkDmFY0i4vEF1/A3Lmw115he8AAeOmlZG1yHMeJi7IVegSQ9ADwMrCHpI8k\n/SI6dIOkKZImE5zQJXFeRz5Mngzt20OzaCBxhx3gj39M1ibHcZy4cG2umFi0CD74AA48MGyvWAE7\n7wxPPw377ZeISY7jOFmRzzCXO5MScuON8Oab8MADSVviOI6TGXcmtSg3Z/LFF7DLLvD666GX4jiO\nU46U4wS8k0aLFnD++TB+fNKWOLny8svhJcBxnLpxZ1Jirr0WevdO2gonF558Ek4+GZZEWgwrVsD1\n10MZdXodJ3EqVeixpaRRkmZKGlmuGfBO5XPPPXDOOcGhHHts2LfhhnDbbfDf/yZrm+OUE7E6k0jo\n8a/AcUA7oJekvdLLmNlNZra/mXUArgJeTMuCvyc6tzZXAs+b2Z7A6Oi8suGii+Cpp5K2wikEs9D7\nuPZaePFFOOigtcckOOYYeP75xMxznLKjUoUeT2LtmvFDCHkqZcPo0SGvxKlcpkyBhx4K81t77rn+\n8a5d4bnnSm+X45QrlSr0uJ2ZLQAws/nAdgXaWTSWL4f33nMJlUpn333hjTfgO3X+WuHoo0OPZfXq\nkprlOGVLpQo91ibjVGiphR7fegu+970wrt4Q/fqFUOGf/zxWk5w8aVbPX0erVrDjjmGZgY4dS2eT\nsz7/+hcccQRsv33DZZ26abRCj5JmAF3MbEGk0zXGzL5XR50lzzO57bbgUAYPbrjsqFFwySUwdSo0\n8bi6imPsWNh9dx/STJJly2CnnWDGDHcmCxfCttuGOb1CKcc8k1iEHqM6+kTfz8hwXiK8+Sb8z/9k\nV7ZrV2je3Cfrk+b99/PL/Tn8cHckSTNsmPdKUnTtChMmJNd+pQo9DgS6SppJWBL4+jivIxduvz37\nYSsJfvMbGLheP624LF0K//hHvG00xMqV8OCD5Zeb8dZb0LkzvP120pY4+XD33XDmmUlbkTyzZ8P8\n+etGHZYal1NJmNWrw7onQ4fCoYfG08Y330Dr1iFCqU2beNpoiOnToV07uPhiGDSoOF3xQnnpJejZ\nE/76Vzj11KStcXJlxowQCPHRR/XPbzUGBg0K96NYL43lOMzlNECzZtC3bxhqiYuNNoJTTgk9g6TY\ne++QQf7SS/C73yVnR4pHHgmO5KGH3JFUKnffHUYBUo7k738PD9TGyKOPwo9+lKwN3jOpQv75z9AL\nueGGtfteegkuvDAM6yTJp59Cly5w2mnJOZV58+CQQ4JD6dCh8PrMyqOn1dj48MMQNZmat7r0UmjZ\nEq65Jlm7Ss3ChWF0Y/788OJYDLxn4vD443D55fCLX6y7v3Pn0DOYMqXu80rFNtuEzPFhw0I+ThK0\nbg3vvFM8R9K+ffhDdkpL27brBkB06xZ+/42N+fOD6kaxHEm+lLs2V53nSuoraU50ziRJx8d9Hdnw\n2WfJtj9mDJx9NjzxRMh1SadJE/jpT+H++5OxLZ3tt4dJk2DXXZOzoXhvcLDbbvDCC8Wpz8mfww4L\nLyjz5iVtSWlp3z7I/iRN3HkmTYBZhIireYRQ4dPM7J0M5U8ELjazY+o7V1JfYJmZDWqg/ZINc61Y\nEbrYixfDxhuXpMl1eP11+OEPwxt/przMefPg66+TfYhXI7fdFrLl77knaUucn/40hAr/8pcNl3Uy\nU47DXIVoczV0blmNUk+bFh7ShToSs/wkOgYMgLvuyuxIIAzvlNqR/OUvIbkvKUaNij+Pp2vXMHTX\nCKfnyo7GOtRVDpSzNldD514gabKkO8tBgn7ixOyTFeujf/+gVpsrjz4a/pDKidWrw7W0apVd+WIO\nE5rBTTdBnz5hUbI42X33MNw1c2a87Tjh/3XChMyO+4c/zO/vxymccorOzkWb63agv5mZpOuAQcBZ\ndRUslTbXpEnFmdDt2ROOOgr+939z6+WUoxzLqFFheeK6VHdrs3w57Lcf3HEH/OAHhbX79ddh7uid\nd8KDZ6edCquvISQ47rigfrDXXg2Xd/Jn3Dg499zMSaabb+4iq/lQDG0uzCy2D9AJeDZt+0rgigxl\nHyHMieR0LtAWmJKhTisVBxxgNm5ccerq3t3sttuKU1eSnHqq2d/+ln35V14x23Zbs+efz7/NDz80\n69DB7PTTzb76Kv96cmXNmtK11Zjp08fsxhuTtqI8GDvW7NZb46k7enbm9LwvZ22ujOdG4o4pegDT\nYrI/K1Jd7n33LU59V1wRhmgyzZ0sX16Y9LlZ/KsELlkCI0fmlhDYqRMMHx5yUMaNy6/defPg9NND\nrs0mm+RXRz6UY8+w2vjiCxgxAn72s6QtKQ8eeCD0wsuFstXmynRudPgGSVMkTSY4oUvivI6GkEI0\n1WabFae+Qw4J62gMH77+sRUrwnrkhcgmfPklHHhgSCCMi2eeCRPTW22V23mHHx7+SHr0CPc0Vzp1\nCkOEnkRYfQwbBkcemf0cXDVTUwOPPRaeBeWCZ8CXKaNHw6JF8JOfrN23enV4azcLazgUokfUq1d4\ncJ93XuG21oVZ6EFtuml+5z/5ZMhuTq277jgHHwxXXw0nnthwWbPQO871ZaZSmDAhzAtOi2lMJrbQ\nYEm7Smoefe8i6cJUYqETD0cdta4jMQsTj0uXhjf3QoXtevcOQ0FxIeXvSCA8MBpyJMuX519/Y2fq\nVLjuuqStyJ6amqDDdXyW6clvvBHyTaqVESPKq1cC2Q9zPQyskbQbMBjYEXggNqucdTALEinTp4cf\nUfPmhdd57LFh3mT27MLrSoLXXw+RU+Um7DdmTBhGLHfuvjsMmVYKTZqEXnS2L1EdOgTNqjgFVJPk\nsceSF3asTbbOpMbMVgM/Am41s8sBXxaoRKxcGVaUe+qp4s3LbLBB6PmUg7xKrtx3X8gnuOWW9WVj\nkqZ//yCqWc6sXBn+3884I2lL4qNp0/AbeeKJpC2Jh+efL04qQjHJ1pmsktSLsKrhk9G+DeIxqbJY\nsABefTXeNpo3D/LaxR7/PeecIJZXKTz7LJx/fsj2HzOm/Lr5sDYbvpx55pmQ+7PbbvDVV/GNuydN\nt27V60zatCm/IJNsnckvgIOBP5jZ+5J2BoZmc2JMQo8tJY2SNFPSyCQz4J99NkiGVCLt22e/KmS2\nPPxwGF6IgxkzQujvq6+GhbbKkWOOgeeeS9qK+rn33qAMADBrVoicq0ZSy9guXZq0JY2DnKO5JLUE\ndjSzBsXMYxR6HAgsNrMbIifT0syurKO+2KO5LrwwZFhfdlmszVQEX30V3pimT2+8a6OvWQPbbhsy\ntMvxHixbBt/9bphLaNEizMe1bh3yespRAHT58hDVl2/AycUXhx54ub58lCtxRnO9KKmFpK2AScA/\nJNWr2BsRl9DjScCQ6PsQQp5KIkyaVBxNrmpgxIgQvlmOD9FS0bRpyIUoV0n6zTeHDz5Yq1cmhQip\nZ55J1KyM3HxzSOLNl7/8xR1Jqch2mGsLM/uCkG1+n5l1BI7J4ry4hB5bmdkCADObD2yX5XUUlTVr\nwsqF+++fROvlx5AhxR82q0TOPTe87Zcrm2++7vYJJ5SnMzELUWfpIfKNnSlTwnOnHMm289hM0g7A\nqcDVMdmSi9BjbTKOZcUp9DhrVsjG3bIKMm4KXXr244+DcrLLf1deomXXriEB7ptvkl+tL53//CcE\nnxx4YNKWlAeffRYWAJs/v/hSQcUQeszWmfQHRgLjzex1SbsA2ag7zQXSNVvbRPvq4jTWDnE1dO58\nSa3MbEGk05VxyjfdmRSbmpoQXVTprFwZhupefnn9t9Zsuf9++PGPk1kYzCmMli2DBM3nn4dVMMuF\nu+6Cs84qv6ilpHjyyZDMHIfmXO0X7WvzWLox7pUWmwIzCZPonwCvAb3SNLZS5bYAZgNtUvpc9Z0b\nTcB/ZmYDk56Arxa6dQuijPmK6L33Xvij32WX4trlNE6++CKErc+aFQIanBB1d9JJpckPinMCvo2k\nEZIWRp+HJbVp6LwYhR4HAl0lpZyNL4dTIIXKq+y6qzuScmbixCAxUiksWBAisYrlSG66KX6l7DhZ\nvjwEdWSjS5YUWfVMJD1HkE9J5Zb0Bn5qZl1jtK1gvGeSPcuXB6XixhzWW8306BEeRGeembQlyXDe\neWGhtt/8JmlL8uPxx0Nk2ujRpWkvzjXgtzWze8xsdfS5F/DOZxWxySYho/yhh5K2pHq45pr8ZPSL\nzaJFQTGgZ8+kLUmO7t0rOxt+001DTls5k60zWSypt6Sm0ac3sDhOw5zS07t3+QknVjIrVsDTTydt\nBTz4YOiV5BtcUQ0ceWQIq41zDZ84Ofro8pQPSidbZ3ImISx4PmEy/MdAn5hsqgiGDQvj0NXE0UfD\n4MG5nfPee2tXmnTWpVykVdLlU+qjpiasc/PNN3FbVHo22ij8vsvBuVcrWTmTKAu9u5lta2bbmdnJ\nwCkx21YU4lrW8o47Kvctp1h8+mlQLq0EyfUk6NwZJk8OkUlJ8dZbsHhxeDNviCZN4KOPYOzY+O1K\ngm7dPA8qTgpZtvfSbAo1JPQYlekSiT1OkzQmbf9FkfjjVEkXpe3vK2lOJA45SVLGJXOGZiVHmRtm\nQUal3CSgS40Pn9TPxhtDx47JStLvvHNY/jnbNeqTllY5/fT4VLhPOQVuuCGeup3CnEmDM/2RWONf\ngeOAdkAvSXvVKrMFcBtwopntA/SM9rcDzgIOAPYDToySJVMMMrMO0efZTDb86U+h+15M3nsvaBs1\n9vj3IUOqe02MYpD0UFeLFrllkCcprTJnDowcCd//fjz1t2jh4etxUogzyWakPBuhx9OBh81sLoCZ\npQaPvge8amYropyTlwjaYCmyCltr0aL4URzeKwmquPPnh3FoJzPnnQd/+EPSVmRPhw5h7fQkVigc\nMiQkzsaR4V2p/PnPoWdZCdTrTCQtk/RFHZ9lQDZSdtkIPe4BbCVpjKTXJaVysKcBnaO1SzYBfkBY\nLjjFBZImS7qzvvVMLr8c7rwzC0tzYOLE6lYKrqkJ8hr1Let6330h+qtp09LZVYlsuWVlDQM2aQLH\nHVf63klNTRB1bKx5MJm4++7yFg1Np15tLjMrxZ9BM6ADcBSwKfCKpFfS1i15DvgSeBNI6WXeDvQ3\nM5N0HTCIMCS2HtOm9aN9e+jXr3hCjyedBFtvXXA1ZUuTJqH39fTTmdeZ3mOPIDrnVB833ghblHi5\nubFjQy7FAQeUtt1y5t13Q5BLp07xt1UMoce4tbk6Af3M7Pho+0rAzGxgWpkrgI3M7Npo+07gGTN7\nuFZdfwA+NrM7au1vCzxhZu3raN8z4PPkrruCM3n44YbLOuXH7NlhobINN0zakuy45ZbQyy2FcGpN\nTUjkbNUq/rYKISUB8/e/l77tODPg8+V1YDdJbSVtSFAGrh2c9xhwWJQMuQnQEZgBIGnb6N+dgB8R\nJF2IlIJT9CAMiTlF5JRTwlrmS5YkbYmTDz/+cZBwrxQuvLB0Ctxjx4Yw4XJnxIjMIwPlSJ6LYWaH\nma2RlBJrbALclRJ6DIdtcDScNRKYQhjGGmxm06MqHo5Wd1wF/DpaoAvgBkn7ATXAB8C5cV5HY2TL\nLcO6HMOHh2VPncJYtiyElKdWOIyTXHJLGiOHHhqGkObNK9/5iM8/h5kzK+v/MNZhrqTxYa7CeOyx\nEE1S4FCqQ1h9ce+94aKLGi5bKJdeGuYfBgyIv61KpVevsDZIOb8oLV+eXGRbOQ5zlRXjx4foCCc7\nTjhhfeFH9835Uap8k1WrwkJlxcj/+fTTZLP346R79/LPhq+0EOlG5UxatoTf/jZ/iRWz8IBdtqy4\ndpUrG2647sp7s2eHBDh3KLlz5JFhDmPVqnjbeeaZEGm3226F13XJJUHloBo5/vigTLB8edKWVA+N\nypnsvXcIPcxXYmXOHHjzTdhss+LaVSkMHQoHH+zLqObDNtuEB3xcUiEpWraE3/2uOHXFLa0yejRc\ndll89ddHy5ZhSeBPPkmm/WokdmdSRG2uC9P2t5Q0StJMSSPrS1qszeWX5y+xksp8b4wPU7OQqOjy\nKfnTtWv8Q12dO4ekw2Jw3HFhHZSVK4tTXzpmcOWVyeaV/PnPYYVQpzjE6kyKrM3VLU2b60rgeTPb\nExgNXJWtTYcfnr/ESrVnvtfHuHFBxruxXn8xOPFEaBZr/GRx2WYb2Guv8H9fbB55BFavDvIpzlpW\nrIBRo5K2Ij/i7pnEpc11EjAk+j6EsIZ8VkihdzJsWO4XM3Fi49TkMgs9kt69G2evrFgcdlhYfbGS\niEP4cfVquPpq+OMfs1czbiyMHg39+ydtRX7E/V8ZlzZXKzNbAGBm84HtcjHqlFPCkE2uTJrUON/M\npTBEk80CS051cdJJxc+Nufde2GGHkMfkrMujj1ZWomI65dDpzkebqzYZ44v69ev37feUNlc+4oRm\nIRpnxx0bLluNJCHp4GTPmjXxiG7uv3/4FJMlS+D6672XW5s1a0Ju1/jxpW+70WpzSZoBdDGzBZG0\nyhgz+14d7XvSolP1rFoV5jZefTXMczi58fvfh8iutm2Ts2HQIPjnP8Ow3xtvJGdHinJMWoxFmyuq\no0/0/YyoDsdplDzzTBg2ckeSHx9/HF8C48qVMH16EEwdMCDzqpuHHgq33VbZahOxOpNo4jylzfU2\n8FBKm0vSL6My7wApba4JrK/NNY3gLNK1uQYCXSXNBI4Gro/zOhynmIwaFUJui8W99/p8ViF07178\nBfTuuy/ktbVoASefHLaXL8+co9axY8jhquQctkavzTVyZBB9K5ViqeMMHhzeUO+/v/C6Fi2C3XeH\nDz8s/Rok1cKXXwbBxwkTQjLjDjusX+a554IU05Il63769Anrv9Tm/feDUsYee4SQ+kojn2GuRu9M\nZsyALl3ggw9g443rLrNmTZgs9DBGpxjMnRtkaXbYIUjF9+yZv/zJLbfAa6+F8fY4eeWVMFxzVp1L\n0DXMzJmw557FtamYnHFGGGI6++y6w7enT4fJk4OzSf9suWXlrBmTC+5MapHtBHy3biGh7NwMQvZP\nPw133FH+wnBO5bBmTVhXY/jwMJ4+fHh+K1dec01Qv41bqnzChKCwO3Vq7ufOmgWHHBK03Uohwe8U\njjuTWmTrTMaODW8k77xTd+9jwIDQFR44cP1jjlMoa6KA9zhCe4vFmjVhZcI338w9PP7UU0N48VVZ\n61Q4SVOO0VwVQefOobuaqeeR0uRynDho2rRuR7JwIey7L1x7bRhmSZKmTUOS4bPP5nbeG2+EvIlS\nrOPiJEu5Cz1eEu2bIun+KLwYSX0lzZE0KfocX5iNQb00kwhfY9bkcpJjm23g9tvDRO+xx0K7dtCv\nX5jnS4J8pFWuuioMxVXa2hxO7sSdtNgEmEUI351HyDs5LQoHTpXZAngZONbM5kraxsw+ldQaGAfs\nZWYrJf0LeMrM7pPUF1hmZoMaaD/rpEWzujNyFy0Kk6NLlvgEvJMcNTUhKfHf/w5RW337lt6GhQtD\ndNLChdlNOr/wAvzqV6FXtcEG8dvnFI98hrnillP5VugRQFJK6PGdtDKZhB4BmgKbSqoBNiE4pBRF\nFWPIJO38SjWAAAAIpklEQVTw7rshocgdiZMkTZqEPISDD07Ohu22C/kx2c7tdOoU5EHckTQOylbo\n0czmAX8CPgLmAp+b2fNp510gabKkO3NZzyRXDj4Ynnoqrtodp7LYf//sncmmm4bEPadxULZCj8Cn\nhF5MW2ApMFzS6Wb2AHA70N/MTNJ1wCDC2ifrUZfQY664IJ3jONVMVQs9EoaxjjOzc6L9PwM6mtkF\ntdpoCzxhZu3raN+FHh3HcXKkHEODCxF6/AjoJGkjSSJM4qcEILdPO78HYe2TojF8ONx0UzFrdBzH\nqW7KVujRzF4DhhPWMXmL0FMZHFV9QxQuPBk4ArikmHbvs0/Q2/n662LW6jjVw+ef173/nnvg5ZdL\na4tTHngGfAa6dw9rRFxxBWy9dZENc5wKZvHiIC65YMG6kVqffRb0t8aPDyHETuVSjsNcFctll4Xe\nyZNPJm2J45QXW28NO+8cxB/TGTgQevRwR9JYKYdorrKkc+cgnnf44Ulb4jjlRyobPvX3MWcO3Hkn\nTJmSrF1Ocvgwl+M4OTN+fFgDaPLksP3LX8JWW4W13Z3Kpxwz4B3HqUI6doSPPoJ588Kw17Rpntzb\n2KlUoceWkkZJmilpZJwZ8M5aCk1qctZS6feyWTM488zgUJo3DxFcLVsmZ0+l389qIFZnEgk9/hU4\nDmgH9JK0V60yWwC3ASea2T5Az2h/a+D/AR2ihMRmhDwVgCuB581sT2A04CsllAD/gy0e1XAvb7op\n6G+VA9VwPyuduHsm3wo9mtkqICX0mE42Qo/NCEKPc6P9JwFDou9DgJNjst9xHMfJgkoTenwhOmc7\nM1sQlZsPbBfjNTiO4zgNYWaxfYBTCBntqe3ewC21ytxKWM9kI2BrwvonuwFbAi8AWxF6KCOA06Nz\nPqtVx+IM7Zt//OMf//gn90+uz/u4o7nmAjulbbdh7VBVijnAp2b2DfCNpLHAvgT5lNlm9hmApEeA\nQ4AHgAWSWpnZgkina2Fdjeca2uY4juPkR0UKPUZ19Im+nxHV4TiO4yRErD0TM1sjKSX02AS4KyX0\nGA7bYDN7R1JK6HENkdAjgKSU0OOq6N+U0ONAYJikM4EPgVPjvA7HcRynfqo6A95xHMcpDVUp9JhN\noqSTPZI+kPRWlFj6WtL2VBqS7pK0QNKUtH2eeJsnGe5nX0lzJE2KPscnaWOlIKmNpNGS3pY0VdKF\n0f6cf59V50yySZR0cqYG6GJm+5vZQUkbU4HcQ/g9puOJt/lT1/0EGGRmHaLPs6U2qkJZDVxqZu2A\ng4Hzo+dlzr/PqnMmZJco6eSGqM7fSkkws3HAklq7PfE2TzLcTwi/UycHzGy+mU2Ovn9JCHJqQx6/\nz2p8QGSTKOnkhgHPRUml5yRtTJXgibfF5wJJkyXd6cOGuSPpu8B+hBVvW+X6+6xGZ+IUn0PNrAPw\nA0I3+LCkDapCPBKmMG4HdjGz/YD5wKCE7akoJG1GWCb9oqiHUvv32ODvsxqdSTaJkk4OmNkn0b+L\nCEoEPm9SOAsktQKoL/HWyQ4zW5S2eNE/gAOTtKeSiLQPhwNDzSyVs5fz77ManUk2iZJOlkjaJHpr\nQdKmwLHAtGStqkjEumP6nnhbGOvcz+iBl6IH/hvNhbuB6WZ2c9q+nH+fVZlnEoUF3szaRElf/y1P\nJO1M6I0YIcn1fr+fuSHpAaALQXtuAdAXeBT4N7AjUeKtmX2elI2VRIb7eSRhvL8G+AA4NzXm72RG\n0qHAWGAqa3W5fgu8Bgwjh99nVToTx3Ecp7RU4zCX4ziOU2LcmTiO4zgF487EcRzHKRh3Jo7jOE7B\nuDNxHMdxCsadieM4jlMw7kycqkBSjaT70rabSlok6fFou5uk3zRQxw6ShkXfz5B0a442NKisKuke\nST1yqbeYSBojqUNS7TvVizsTp1r4CthHUvNouytpgp9m9oSZ3VBfBWb2iZmlr9qZaxLWb3MsX1FI\napq0DU754s7EqSaeBn4Yfe8FPJg6kN7TiHoHN0saL+ndVE8hkuCZmlbfTtGb/ExJv0+ra0SkoDxV\n0tnRvj8CG0cLMw2N9v08bVGxIWn1HlG77XQiO6ZLGixpmqRnU04yvWchaWtJ76dd34hoQaPZks6X\ndElkz8uStkxr4ueRTVMkHRidv0m06NQESRMldUur9zFJLwDP5/5f4jQW3Jk41YIR1q7pFT142wOv\n1lEmxfZmdijQDRiYocyBwI+AfYGeacNDvzCzA6PjF0lqaWZXAcujhZl+JmlvQk+li5ntD1yURdvp\n7Abcamb7AEuBU+q57hTtCOtOHAT8AfgyUnueAPw8rdzGkU3nE3SZAK4GXjCzTsBRwE2SNo6O7Q/0\nMLMjM9jgOO5MnOrBzKYB3yX0Sp6i/sWSHo3OmUHmtRqeM7PPzewb4BEgJb1/saTJhId0G2D3aH96\ne0cB/zazJVE76bpG2bT9vpmlekkTo+tqiDFmttzMPgU+B56M9k+tdf6DUfv/ATaX1IIg4HmlpDeB\nF4ENWau+/ZyZLc2ifacR0yxpAxynyDwO3EgQAtymnnIr0r5ncjrrrekg6QiCo+hoZiskjQE2ytHG\nbNpOL7MmrY3VrH0JrN1u+jmWtl3Dun/rda1VIeAUM/tv+gFJnQjzUY5TL94zcaqF1EP5buBaM3s7\nj3Nr01XSltFwz8nAeGALYEnkSPYCOqWVX5k2ST2aMDS2FYCkljm2nWn/B8AB0feeGco0xE8imw4D\nlprZMmAkcOG3jUv75Vm300hxZ+JUCwZgZnPN7K/ZlK1nO8VrhOGtyYQhq0nAs8AGkt4G/g94Ja38\nYGCqpKFmNj06/lI0dPSnHNvOtP8m4DxJE4GtMpRpqN5vJE0irE54ZrR/AOG6pkiaBvSvp27HWQ+X\noHccx3EKxnsmjuM4TsG4M3Ecx3EKxp2J4ziOUzDuTBzHcZyCcWfiOI7jFIw7E8dxHKdg3Jk4juM4\nBfP/AYjIoLThXwd4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa8ea668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmcFNW1x78/RFAUxN0nuCuuSVzRqMhoFHHfFzRxTTTG\n3egTTZ6QqM/daPSpD0WfcYFgXFiMituIEVBcUFQElEVZZBMRAwLjnPfHrYaepqeneqa7q3vmfD+f\n+kxV9V1OV9fUqXvvWWRmOI7jOE5DtEpaAMdxHKcycIXhOI7jxMIVhuM4jhMLVxiO4zhOLFxhOI7j\nOLFwheE4juPEwhVGhSDpfkl/yLespO6SviqudCv6nSLpoFL0VWlIqpW0danrNpX0vvO5B7O0s0jS\nloWUzSk9rjASRtJUST9IWi/j/AfRP+vmAGZ2gZndGKfNLGUb5WwjaYtIBr9PaPL1aIrDU711JVVL\nWiLpO0lzJD0taeMm9FVv33HvQUmvSzqnTiNm7c1sagHlSvU1VdLi6Psviv7+tdD9OAF/ECSPAVOA\nXqkTknYB1qRpD5lCoEgGFb0jabVi91EgGvubNOUa5qprwO/MrAPQBegI/CVrI41TdEX/7ZuIAUeY\nWYdIKXUws0uyFcx2j+V731XQfVoUXGGUB48BZ6Ydnwk8ml5A0iOS/hztd5f0laQrJM2WNEPSWdnK\nrjylayTNlTRZ0mlpHxwu6X1JCyVNk9Qnrd4b0d9voze3vaM6v5H0aXTuY0m7ptXZTdKHkhZIGiCp\nTbYvLOlMSf+SdKekuUAfSX0kPZZWps4bffTm+ueo3neSXswcmaXV/VTS4WnHq0Vv4LtKaivpMUnz\nIjnflrRhtnbiImkvSSOj9mZIukdS64xiR0j6IpLj1oz650Qyz5f0QmpkGbd7ADP7Fnga2CVq8xFJ\n90l6XtIioEpSG0m3R7/1rOjztmlyXCVppqTpks4mTUFm3leSjolGwgslTZLUQ9INQDfg3vS3fdWd\n2uog6W/RdZiitGmu6L54U9Jtkr6JrlfPON9/lZN177F5hHss2zlJ+qPCaOVrSf8nqUPURuoePEfS\nNODVuD9Kc8QVRnkwGmgvafvo4XgK8Di53+42AdoDmwK/Bv5H0jo5yq4XlT0L6Cdpu+iz74Ffmdk6\nwBHAbyUdHX12QPS3Q/Tm9rakk4DrgF9Gb7VHA/PT+joJ6AFsBfws6q8+9gY+BzYGUlMdmW/wmce9\nCAp1Q6AtcGU9bT8JnJZ23BOYa2Zjo/odgE6E6/JbYEkOOePwI3BZ1N7PgYOA32WUORbYPdqOUTRt\nI+kYoHf0+YbAm8CAfAWQtAFwAvB+2ulewPVm1h54C7gF2Bb4afS3E+H3JHowXwH8AtgOODhHX10J\nLzW/j+6dA4CpZvbHSP6LMt7203/Hewn37pZAFXBGpJxSdAXGA+sDtwH987kOGaTusY1YeY9lnjsb\nOAPoDmwdyXZvRjsHADsAhzZBlorHFUb5kBplHEL4Z5nZQPllhAfBj2b2AuHBv309ZQ34LzNbbmYj\ngOeBkwHMbISZfRLtfwwMJPzjpJOuuM4FbjWz96M6k80sfVH9bjObHb3tDgXSRx+ZzDCz+8ys1syW\nNvB9UzxiZl9E5QflaH8AcLSkNaLjXqx8CC8nPIy6WOADM/s+Zv9ZMbP3zeydqL0vgX6seh1vNrOF\nZjYduIuV05DnAzeZ2UQzqwVuBnaVtFnM7u+R9A3wAeG++X3aZ4PNbHQk41LgN8DlkRz/jvpKyXES\n4fqON7MlQN8cfZ4D9Dez16K2Z5nZxBzlBSumxU4BepvZYjObBtwB/Cqt7DQze9hCoLtHgU0kbZSj\n7eei0ciC6O+5aZ9lu8cyz50G3Glm08xsMXANcKpWTuEZ0MfMluRxnzZLMofMTnI8DowgvJn/LUb5\n+dHDJcViYO16yi4wsx/SjqcRRhsoTDPdRJjGaBNtT+XodzPgixyfz86Q6T9ylG2M9dbXGe1n/c5m\n9oWkT4GjJA0jjISuiz5+DOgMDIxGZY8DfzCzHxshDwDRiO1OYE/C+lNr4L2MYtPT9lf8BsAWwN2S\n7kg1R3hIdSLeNbrYzB6u57MV9aNpt3bAe9KKd4BWrHwh2BR4N0PG+ka5mxFePPJlA8K1+TKjn05p\nxyt+YzNboiDs2sCceto8xsxer+ezbNcv89ymkQzp8rQmjHxTTMfxEUa5EL2VTgEOA54pcPPrSloz\n7XhzVo5gngCeAzqZWUfgf1n5kMi2wPsVsE2B5Mps/9+EB1qKXMomDgMJb4/HAJ+Y2WQAM6sxs+vN\nbGdgX+AowpREU7ifMDLcJrqOf2DVh236iGELVv4GXwHnm9l60bauma2dGhk0kfRrPI+gZHdO66tj\nNKUEMCuLjPUt8ue6D3IZBswjjPC2yOhnRo46DdGQUUBD52ZmkWc5dV9+kjZAKQtcYZQX5wAHRdMB\nhUTAnyStLqkbYa1iUPTZ2oQRyPJoXjp93n8uUEvdB8NDwJWSdgeQtE0eUycNMRY4QNJm0Zt/7ya2\nN5CwnnIBYU0DAElVknaJphy+JzwcarM3sQoC1ogWzlObCPPe35nZYkk7RH1mcpWkjtH1uiSSD+AB\n4FpJO0XyrSPpxPy/bm6iKZ4Hgbui0QaSOknqERUZBJwlaUdJ7Vg5IstGf+BsSQdGi8abSkpNic4m\nrAVkk6E26udGSWtL2gK4nDDqS4oBwOWStpS0NmFdY2DaCL7cLcVKhiuM5Em3c5+SWhvI/CyfdrIw\nC1hAeJN6jPA2Oyn67HfA9ZIWAn8E/p4mzxLCP89b0dxwVzP7R3TuSUnfAc8SFnrzlXfVL2D2StT/\nR8AYwhpInSJ5tvc1MArYh7TvRTAC+AewEPgEeJ3ogaXgnHZfrmaBRYQ39SXR3wMJ6wanR9fkf1mp\nDNLrDSZMU70ffbeHIzmfI6wlDJT0LeH798yom0uefD67mrDgOzrqazjBHBcze5GwtvIaMJEcFkFm\nNoawWHwX4TpWE0auAHcDJylYfN2VRZZLCNdtMmEa9nEzeyTP75HO0MgiK7U93UD5TB4m/P4jCNOt\niyMZ4/bfYlCxEyhFlhd3EZRTfzO7JUuZKoLt+OoES5YD49Z1HMdxSkNRFUY05J9IMNObSXhrPNXM\nPksrsw4wEuhhZjMkbWBm8+LUdRzHcUpHsaekugKTInO15YRh+jEZZU4DnjazGQBmNi+Puo7jOE6J\nyKkwFLxj6zNXi0OmWeB06prPQZg/XU/Bi3eMpF/lUddxHMcpETn9MMzsx8gtfh0zW1hEGXYneMau\nBYySNCqfBiT5opTjOE6emFleFmBxHPe+B8ZJeplgJ5/qKGuArwxmsNJyAoKzVKa99XRgXuRY9oOk\nEYSQEnHqrqDYi/cthb59+9K3b9+kxUiORYvguONg7bXhiSdgrbWa1FyLv54Fxq9n4Uhz3oxNHIXx\nDI13JBsDbBvZWs8CTiUtKmvEYEJog9UIsYH2JnjMTohR13EKx5w5cPjhsMcecN99sFqLDkzqOKvQ\noMIws0cVIo52iU5NiBahGySa0rqIYOudMo0dL+n88LH1M7PPJL1EsD3/EehnZp8CZKub7xd0nFiY\nwZFHwhFHQN++0Ii3L8dp7jRoVhv5SDwKTCV4PG4GnGkhiF1ZIMl8SqowVFdXU1VVlbQYyTBrFvxH\nU6OR1KVFX88i4NezcEjKew0jjsJ4DzjNzCZEx12AAWa2R6MlLTCuMBzHcfKjMQojjh/G6illARCF\nMF49X+EcpyKZPh3mzWu4nOO0AOIojHclPRQFbKuS9CB1QyA7TuUxPuZy2A03BGspx3FiKYwLgE8J\nwbguifazReJ0nPLHDK6/Ho4+GpbECArcrRv861/Fl8txKoCcaxiRqevfzOz00omUP76G4cTixx/h\n0kvhzTfhxRfjLXBPmwZ77x0WxN1yymlGFHwNI8pAtkVkVus4lcvSpXDaafDxxzBiRHxrqM03h9VX\nhy9yJRl0nJZBHMe9yYR8CEOo6+l9Z9GkcpxCc9ZZUFMTRhZrrNFg8RVIsP/+YVpq222LJp7jVAJx\nFMYX0daKkFXMcSqPG26ALbdsnPf2CSeEEYrjtHDirGHcYmZXlk6k/PE1DMdxnPwo1hrGfk2SynEc\nx2kWxJmSGhutXzxF3TWMxgYkdJziYuYWTY5TBOIojDWA+YR8FSmMxkewdZzictFFcMABcMopSUvi\nOM2Koub0LhW+huGsYPz4oCwmTID11ktaGscpWwq6hiFpUNr+LRmfDc9fPMcpAb17h60YyuLmm+H7\n7wvfruNUCLkWvbdL2z8k47MNiyCL4zSNESPgo4/ClFQxGDYMRo8uTtuOUwHkUhi55nh8/scpL2pr\n4cor4cYboW3b4vSRcuBznBZKrkXvdpJ2IyiVNaN9RduapRDOcWKzdCkceyycemrx+ujWDf7yl+K1\n7zhlTr2L3pJez1XRzA6M1YHUE7iLlWlWM9dDuhPyek+OTj1jZjdEn00FFgK1wHIz61pPH77o7RSf\nBQtCbKlvvgnxpRyngmnMone9I4y4CqEBgVoB9wK/AGYCYyQNNrPPMoqOMLOjszRRC1SZ2YKmyuI4\nTWbddUN4kbFjYa+9kpbGcUpOnHwYTaErMMnMppnZcmAgcEyWcvVpOVF8GR0nPnffDZ07Jy2F4yRC\nsR/GnYCv0o6nR+cy+bmksZKel7RT2nkDXpY0RtJviimo48TioIPih0Z3CsusWTBzZtJStGjieHoX\nm/eAzc1ssaTDgOeALtFn+5nZLEkbEhTHeDPLaqbSt2/fFftVVVVUVVUVV2oneb78MjxA9tknaUmc\nUvDAA0Fp9OuXtCQVSXV1NdXV1U1qI5ant6ROwBakKRgzGxGj3j5AXzPrGR33DlXrLnxn1JkC7GFm\n32Sc7wMsypaHwxe9Wyinnw7bbQdpLwtOM2bePOjSJSTB2nTTpKWpeBqz6N2gwoi8vE8h5PL+MTpt\n9SxSZ9ZdDZhAWPSeBbwD9DKz8WllNjaz2dF+V2CQmW0pqR3Qysy+l7QWMBz4k5mt4mXuCqMF8u67\nIS/3xImw9tpJS+OUiksvhTZt4Lbbkpak4imWwpgA/NTMGpVBJjKrvZuVZrU3SzqfoHT6SboQuABY\nDiwBLjeztyVtBTxLWMdoDTxhZjfX04crjJaEWVhL6NULzjsvaWmcYlFbC60yllm//BJ22w0+/zxY\nrTmNplgK4wXgJDMr2yA6rjBaGM8/D1ddFcKAtE5gGW7GDDjjDHj11dL33VKYPx8OPhiGDl3VKu3s\ns2HrreG//isZ2ZoJBfXDSGMxISfGq8CKUYaZXZKnfM2TpUuLF4rCyc6tt8IttySjLCBYSX3wAXz9\nNWyySTIyNGeWL4eTTw4KI5sJc58+sGRJ6eVyYpnVDgGuB0YSLJpSm/PVV7DTTuHB4dRl5swwCli+\nvPBtDxsGRx5Z+Hbj0qoV7Luvx5UqFpdfDmusEaIDZ2PLLWHHHUsqkhNo8BXNzB6V1IaVpq4TIie8\nls3ixSF20fnn+1tmiqVLw4KkBB07htwUxx4LgwbBWmsVrp/27QvXVmPp1i0ojBNPTFqS5sUDD8Br\nr8GoUbDaaklL42TQ4AhDUhUwCfgf4D5goqQDiixXeWMGZ50FO+8c3qLTzx95ZLDcaYn06ROmiwDa\ntYNnn4UNNwxTC/PnJytbofHItYVn/vwwqhgyBNZZJ2lpnCzEmZK6A+hhZt3N7ADgUKBlh+y8/vpg\nrdGvX93c0RIccgice26w8GhpDB8O++238nj11eGRR0IGvG7dwhRec2HPPcOLweLFSUvSfFh//TAq\n3XbbpCVx6iGOldRHZvbThs4lSUmtpGbPhgMPDBYy2UJE1NaGB+TJJ8MlLcguYM6c4FQ1d272SK53\n3BGmGC67rHHtm9VVzuXA99+7D0jSTJwICxd6MMhGUCyz2ocJUWMfj06dDqxmZuc0SsoiUHKz2pqa\n3BY6EyeGRdG334ZttimdXEnyxBPwj3+EaahCM358UL7Dh5ef0nCS5emnwzTo6NF+b+RJQXN6p3EB\nwcv7kmj7NDrXcmnInLNLl5BX+te/bjlTU8OHQ48exWm7d2/o2dMfCM6qHHdcGGG8njN9j1Mg4lhJ\nLQXujDYnLpdf3rLCYM+fXxyFkcrTPWhQ4dt2kmXAAFhzzWBJ11hatYKrr4abbgre/05RyZVxb5CZ\nnSxpHFlyeLfYNYxyZfHisAh/8cWVY444eTL8859w0UXZP6+tDZFoL7sMTjuttLI5xeXtt4NF4euv\nwy67NK2tZcvC1O+zzwZjBCcWhZ6SujT6eyRwVJatZXDPPWGetNyZMyeMat58M2lJ4tOmDdx/P1x5\nZfapu0GDwvli5uluKosXh1AhTnxmzIATToD+/ZuuLCDcR7//vQckLAH1KgwzmxXt/i7KmLdiA35X\nGvES5qWX4L//uzLeWrbcEm68EZ56KmlJ4tO5c1Bwo0YFv5ZMr/C2beGuu1YNQFdODBlS/wipHFi2\nLGkJ6rJkSZiCuvDCEG24UJx3XrhXnKISx0rqfTPbPeNc8zernTAh+A4880xw0qoEJk6E7t1h+vTK\nmZaC8JZ+8snBdLbQXuHFZvr0ED11zpzyXJTfbz+4/Xb4+c+TliRw7rnwww/w+OPleb1aEAWdkpJ0\nQbR+sYOkj9K2KcC4pgpb1ixYAEcdFRbSCqUsPvggJPwp5lpLly6w0UYwcmTx+igG6V7hgwcnLU1+\ndO4cFNyECUlLsioLFsC4cUGh9e8fzMGT5oor4KGHXFlUKLnG+k8S1ioGU3ftYg8zO70EsiXHeefB\n4YeHt6FCsfPOIVPYY481va0XXgiOcNk49dSgnErF88+HxeumkvIKr8TF7VRcqXLjtdfCCGONNeDe\ne8NCc9LsvHOwjHKaxjffNFymCORaw1hoZlMJyY++SVu/qJG0d6kETIRbbw3D+ELSpk14IF55ZchL\n3FgeeyzM9++7b/bPe/curYf51VeH1JmFoFLfOvffvzyNDdJ9Yw49NBw7lc/o0bD33on4eMVZTbwf\nSE+e9H10LhaSekr6TNJESVdn+by7pG8lvR9tf4xbt2hstVVxci3svnsYvVxwQeOmpu64A/74x2CK\nWN+cdCkfutOnB+W3xx6l67Mc6d4dOnRIWoq6mAWjjUMPDcc9eoTjlsAnn8CTTyYtRXEwCy+d11yT\njDGImeXcgLFZzn3UUL2oXCvgc2ALYHVgLLBDRpnuwJDG1E0raxXDDz+Y7byz2ZNPxq9TW2t21VVm\nO+5o9uWXxZMtXx5+2Ozkk5OWwsnG3Llm++0X7h2zcN+1b282f36ycpWCCRPMNtjAbNGipCUpPM88\nY/aTn5jV1DS5qei52eBzPH2Lo6ImS7pE0urRdikQd9K6KzDJwnTWcmAgcEyWctlei+PWrSzatg3T\nSjvsEL/O/PkwdWqY9thss6KJljcvvVS8cCBO09hgg7Cukhpxtm0b1lqSSCt7xx3hrb9UdOkSAoT2\n61e6PkvB8uVhCvjWWxOzgoyjMH4L7AvMAKYDewPnxWy/E5Ae03p6dC6Tn0saK+l5STvlWbfx1NbC\nmDHwf/9XXOulTHbbLWxx2WCDYG66/vrFkylffvwRXnnFFUYlcfXV+b2oFIKamuDLVOopu9694c47\nQ1Kv5sLs2cE7PjXNmABxYknNAYrpavsesLmZLZZ0GPAcK7P7xaZv374r9quqqqiqqspecMmS8JY1\nZEhI9bnOOiGpfHNj0iT48MPiZYSrqQn/kOU04nFyc0ACec9GjoTNNy/9fbL77sEi6/HHC2vtmCSd\nO4f/uUZSXV1NdXV1k0TIFUvqP83sVkn3kD2WVIOmOJL2AfqaWc/ouHeoarfkqDMF2IOgNGLVje24\nV1MTbtzttw9epkcdBdtt13C9SmTcuPD9pkypXOsjp/K56qpgRvvnP5e+7+rqsDg8alTp+64AGuO4\nl2uEMT76+27jRWIMsK2kLYBZhJFKr/QCkjY2s9nRfleCEvtGUoN168UsTDdlzvO1bg1ffBEcxcqV\n8eOD81rv3k1rZ5ddwrz1mDHQtWthZHNy8/zz4YXkp2UTBCF5hg2Dv/0tmb67dw/Tpk7ByOWHMTT6\n+2i2LU7jZvYjcBEwHPgEGGhm4yWdLym1DnKipI8lfQDcBZySq269nS1bFm6OSy4JZrH1eQyXo7K4\n7rog7+jRYbFu002b3qYEJ50Ukho5pWHUqPKI5fXII/DvfyctBXz+OXz7bXJm11JlhZmpAHJNSQ0l\ny1RUCjMrYOSwpiHJrGPHlVNNRx8d5i8rZSpmxIjwcK+thUcfDV7mhWDs2JBgZvLkyrkWlcyLL8It\ntySbzGfu3JATe9687KlyS8ny5eHe2377ZOWoZKZNC2mAi2DwUtAUrZK6R7vHA5uwMkVrL2C2mV3e\nWEELjSSzWbNgk02SFqXx3H9/sJzaZ5/CtWkWTAwHDKiMiLuVznffQadOwQy6TZtkZBgwAAYOzB2T\n67zzQlyz7t3rL+OUB4cdFl4gL7644E0XNPigmb1hZm8A+5nZKWY2NNpOA7o1VdiCU8nKAoL3dyGV\nBYRRxYAB4Y2zkHTrBl991XC5lkaHDuFav/9+cjKke3fXxyabhHhkTnnzyivB2vH885OWZAVx/DDW\nkrR16kDSVoBPDFYKe+4JHTsWrr3Jk8NN3KmwLjHNhv33Ty4QoVm83Oo9erS8uFKTJgU/lEqhtjZY\nmN10U3Kj1SzECZh0OVAtaTLBI3sLoHxUnlNaXn4ZDjmkvJMaJcm55yaXtOjjj4MJa0Mjyr33Dop/\nzpwQDr8l0KlTcNA980zYaacGiyfOE0+EKMPF8qNqJA0mUAKQ1BZIuYh+ZmZl5T7pOb1LyPHHh+2X\nv0xaEieT6dODs+YRRzRc9rjjwsPo9CJkKqipCes4G29c+Labwo03hrwlSZn5xqWmJviHPf54CE9f\nJAqd0zvVaDvgKuAiM/sQ2FzSkY2U0alkampCjoWDD05aEicbnTvHUxYQpqWK5dA2alRYrC03Lrww\nxGO76abShgLKl9atw/pFEZVFY4kzJfUIIXxHKp72DOApYFixhHKKwLJlITVmU2L6fPppmO6odAMD\nB3796+KE8AcYOjTEPCo3OnaEt96Cnj3h++/DiKNc2WabpCXISpyc3u+a2Z6SPjCz3aJzH5rZz0oi\nYQx8SioG11wT7PKbGqKhpqZ4DxqnebDjjmHaZ6+9kpYkO99+CxMntvgICEWZkgKWSVqTyIlP0jZA\nWa1hODE49tjghdxUxerKwslF0t7dcejYscUri8YSR2H0AV4ENpP0BPAq8J9FlcopPF27hnARn36a\ntCQtg6OPDovQLY2hQ8M6ilvRNUty/qqSBHxG8PY+CxgA7Glm1UWXzCksUrCKKYdYRy2BVq3CfHkp\nmDkz+H+UA61bw6nFzIZQRJYsSa7vt9+Ga69Nrv+Y5FQY0cLAP81svpk9b2bDzGxeiWRzCs2JJ3ow\nwlLRrVuwyCkFw4c3PmDl5MkhBH6huPjiyrSimzIlrL2MG1f6vlN5uisg1UKcceP7ksp09crJi332\nCfGq8o1kmvIgrq0tjlzNkVJ6fMfx7q6PJ5+Ev/61sPJUIlttBTffHJRdqT31Bw+GhQvhjDNK228j\niGMl9RmwHTAV+DfB29vMrGyC/ruVVJGZMCF4d0+b5lFv47JsGay3HsyYEbI6Fova2uAg9957IbNd\nvowZA2edVdqc2+XMyy8HZ8aHHgrrUMVm+fKQu+buu4O5bwkpdAKlFMklkHXKg5deCm+wrizi06ZN\nMCsdNaq4D4IPPgg53xujLCCkMp09OyzQd+5cWNkqkUMOCYmwjj46vPX/6lfF7e/BB8Nvl2Ce7nyo\nV2FIWgP4LbAtMA7ob2Y1pRLMKSOGDw8xeJz8ePppWHfd4vbxzjuNn46CkJXy4IPDb3zOOYWTq5LZ\nay94442gMIrN11/DrbdWzMtYrnwYfweWA28ChwHTzOzSvDuQehIy6bUiKJ2s+byjdZKRwClm9kx0\nbiqwEKgFlptZVuNpn5IqIkuXwoYbwtSpYYrFKT+a6kz58MNBYQwc2Pg2HnwwBPUrw3AWTnYKPSW1\nk5n9JGq4P/BOIwRqBdwL/AKYCYyRNNjMPstS7mbgpYwmaoEqM1uQb99OgRg5MliPuLIoX5rqTHno\noSFYYFO4887yD+rnNJlcVlLLUztNmIrqCkwys2lmthwYCByTpdzFwD+AORnn1YCMTmO54orgldsQ\na68dTP6c5kunTiH3QmOpBO/uQlHTsmflcz2Mfybpu2hbBPw0tS/pu5jtdwLSU7NNj86tQNKmwLFm\ndj9BQaRjwMuSxkj6Tcw+nTgsXRrPJ2OvvUK+ccepj2HDQrDB5u7d/dZbYcpt2rSgIFPb0noiJS1e\nXLdcQ+UrgHrHsma2WolkuAtIT4WVrjT2M7NZkjYkKI7xZpbVSLpv374r9quqqqiqqiqCqM2IE08M\nI4fevZOWpGXw0EPQrh2cdlrSkhSeoUPh0ryXNyuPffcN03e77lo3Jttf/gJnn71q+WuvDUmbMqmv\nfJGprq6murq6SW3ESqDU6MalfYC+ZtYzOu5N8OG4Ja3M5NQusAHB1+M8MxuS0VYfYJGZ3ZmlH1/0\nzpeamjAVMWoUbL11w+WdpjFyZAiZMWFCyIrXVObNgy+/DGaxSbJwIWy2GcyaBWt55uZKoljRapvC\nGGBbSVtIagOcCtRRBGa2dbRtRVjH+J2ZDZHUTtLaAJLWAnoAHxdZ3pZD69Yh65qHCikN++4bAkDe\nfXdh2nvmmfCmmjTt2wfnP1cWLYKiKgwz+xG4CBgOfAIMNLPxks6XdF62Kmn7GwP/kvQBMBoYamYt\nLHN9kTnxxBCWwCkNN90Et98Oc+c2va2mhAOpjzfegNtuy69Oq1aw/faFlcMpW4o6JVUqfEqqkdTU\nhIW5bFn4PvkkzLuXw1tsc+Lii4OTVlPiN9XUwEYbhVD1hcx++OGH4SVi0qTCtemULQWdkkpZQ6VZ\nRi1qhJWUU860bl1/ytYXXqhoa46y5brrQvDHprzgjBkTwkkUOlXuT34CixaFCLaOk4V6FYaZtTez\nDtHWPu07SSdBAAAKY0lEQVS4vZk1ITG0UxEUY8rDCV7z/fs3LRREKrZXoWnVKrT78suFb9tpFsRa\nw5C0v6Szo/0NJG1VXLGcRFm8OFhPHXRQ0pI42dh+ezjllOK03aNHUEhxaIkZBVs4ccKb9wH2BLY3\nsy6Ro91TZlY2QWN8DaPAvPQS3HBD6RIAOeXD7Nmwww5hYT5XyJHPP4cDDgjh2yskcJ5Tl2KZ1R4H\nHE3wj8DMZgLt8xfPKVuWL4fXXlt57NNRLZeNNw6L3w3Fp0rl7nZl0aKIozCWRa/vBit8IpzmhFmw\njpkxIxxff32w5nGKz8SJQWGXE3FyawwdCkcdVXxZnLIijsIYJOl/gY5RPKdXgAeLK5ZTUtq0CQlj\nnn46HLdrBx07JitTS+Gyy6Bfv6SlyI9vv4V334Vf/CJpSZwSE8sPQ9IhBE9rgOFmVlZmFL6GUQCG\nDYNbbvF1i1Lz4YchPtHEifWbOJcbf/97CGX+/PNJS+I0gWKGBhlHSKQ0Itp3mhuHHBKc9WbOTFqS\nlsXPfhZSuN6SNa9YXZ54ojxyTpiFPOBOi6NBhSHp14TkSccDJwKjJXkux+ZG27YhRPUzzyQtScvj\nhhvggQcaNlMdODD8TqWgtrZ+eU491UPet1DimNVOAPY1s/nR8frASDMrmwAyPiVVIN55J3ghH3hg\n0pK0PK69NpiyPljP8uCyZcHpb/JkWH/94svz2WfBUm7aNLeEaqYUOkVrivnAorTjRdE5p7nRNWvK\ndKcU9O4dwnLUx8iRwWGvFMoCVgYUnDAh+GU4DjkUhqQrot3PgbclDSaY1h4DfFQC2Ryn5dChQ+5F\n71L7xkihv+HDXWE4K8i1htE+2r4AnmNl6PHBwJQiy+U4TjpJOFMeemj8MCFOi8DDmztOJTBnDqy7\nLqy+eun6/OYb2HLLsLbStm0wp23Xzh32mglFMauVtKGk2yT9U9Jrqa3xYjqOkzcbbVRaZQGw3npw\n+ukhvhQEB8Pa2tLK4JQVcfwwngA+A7YC/gRMJaRedRynGPzwQ3hQL1mStCRw//0hVMjChSEPx8EH\nJy2RkyBxFMb6ZtYfWG5mb5jZOUDsuNeSekr6TNJESVfnKLeXpOWSjs+3rlM4qqurkxahWdGo67nG\nGiF5VaHyfxeCF1+Ebt0Sz93t92eyxFEYqchosyQdIWk3YL04jUtqBdwLHArsDPSStIrJRVTuZuCl\nfOs6hcX/IQtLo69nIfN/F4IyCTbo92eyxFEYN0haB/g9cCXwEHBZzPa7ApPMbJqZLQcGEsxyM7kY\n+AcwpxF1Haf5sd120KsXXHBBcNpLkpqaMMI48shk5XASp0GFYWbDzGyhmX1sZgea2R7ANjHb7wR8\nlXY8PTq3gigh07Fmdj+gfOo6TrPmuutCBOHnnktWjtVWCxkYO3dOVg4necws7w34Mma5E4B+ace/\nBP6aUWYQ0DXafwQ4Pm7dtM/MN9988823/LZ8n/1xQoNkI67t7gwgPRtL5+hcOnsCAyUJ2AA4TFJN\nzLoAedsSO47jOPnTWIVhMcuNAbaVtAUwCzgV6FWnIbOtU/uSHgGGmtkQSas1VNdxHMcpHbliSS0i\nu2IQsGacxs3sR0kXAcMJ6yX9zWy8pPPDx5aZaswaqhunX8dxHKfwNIvQII7jOE7xiZtxryxxx77C\nImmqpA8lfSDpnaTlqTQk9Zc0W9JHaefWlTRc0gRJL0Um6k4D1HMt+0iaLun9aOuZpIyVhKTOUVin\nTySNk3RJdD6v+7NiFYY79hWFWqDKzHYzM0+OkT+PEO7HdHoDr0QJx14Drim5VJVJtmsJcKeZ7R5t\nL5ZaqAqmBrjCzHYGfg5cGD0v87o/K1Zh4I59xUBU9j2RKGb2L2BBxuljgEej/UeBY0sqVIVSz7WE\n+BaaThpm9rWZjY32vwfGEyxP87o/K/nh4I59hceAlyWNkfSbpIVpJmxkZrMh/NMCGyUsT6VzkaSx\nkh7y6b3GIWlLYFdgNLBxPvdnJSsMp/DsZ2a7A4cThqz7Jy1QM8StTBrPfcDWZrYr8DVwZ8LyVByS\n1iaEYbo0Gmlk3o85789KVhixHfuceJjZrOjvXOBZwrSf0zRmS9oYQNIm1I2X5uSBmc1Ny5T2ILBX\nkvJUGpJaE5TFY2Y2ODqd1/1ZyQpjhVOgpDYEx74hCctUsUhqF719IGktoAfwcbJSVSSi7jz7EOCs\naP9MQopjJx51rmX0QEtxPH5/5svDwKdmlh43P6/7s6L9MCKzurtZ6dh3c8IiVSyStiKMKozg0PmE\nX8/8kPQkUAWsD8wG+gDPAU8BmwHTgJPN7NukZKwU6rmWBxLm3msJidzOT82/O7mRtB8wAhjHylhS\n1wLvEOL5xbo/K1phOI7jOKWjkqekHMdxnBLiCsNxHMeJhSsMx3EcJxauMBzHcZxYuMJwHMdxYuEK\nw3Ecx4mFKwynopBUK+lvacerSZoraUh0fJSk/2ygjf+QNCjaP1PSPXnK0GDEWUmPSDo+n3YLiaTX\nJe2eVP9O88QVhlNp/BvYRVLb6PgQ0oJQmtlQM7s1VwNmNsvMTk4/lacM1+ZZvqKI0iM7ziq4wnAq\nkX8CR0T7vYABqQ/SRwzRW/7dkt6S9HnqjT8KJzMurb3NozfyCZKuS2vr2Shy7zhJv47O3QSsGSXw\neSw6d0Za4qlH09rtntl3OpEcn0rqJ+ljSS+mFGH6CEHS+pKmpH2/Z6OkN5MlXSjp8kiekZI6pnVx\nRiTTR5L2iuq3i5ITjZb0nqSj0todLOlV4JX8fxKnJeAKw6k0jJD7pFf0cP0p8HaWMik2MbP9gKOA\nW+opsxdwHPAz4KS0qZyzzWyv6PNLJa1rZtcAi6MEPr+StBNhxFFlZrsBl8boO51tgXvMbBdgIXBC\nju+dYmdC3oKuwI3A91GU4dHAGWnl1oxkupAQRwjgD8CrZrYPcBBwu6Q1o892A443swPrkcFp4bjC\ncCoOM/sY2JIwunie3El1novqjKf+WP8vm9m3ZvYD8AyQCut+maSxhAdxZ2C76Hx6fwcBT5nZgqif\n9Dg8cfqeYmap0c570fdqiNfNbLGZzQO+BYZF58dl1B8Q9f8m0F5SB0JQyd6SPgCqgTasjPr8spkt\njNG/00JpnbQAjtNIhgC3EQLUbZCj3NK0/foUyyo5ASR1JyiDvc1sqaTXgTXylDFO3+llfkzro4aV\nL3SZ/abXsbTjWur+T2fLdSDgBDOblP6BpH0I60OOUy8+wnAqjdSD92HgT2b2SSPqZnKIpI7R1Myx\nwFvAOsCCSFnsAOyTVn5Z2sLwa4RprPUAJK2bZ9/1nZ8K7Bntn1RPmYY4JZJpf2ChmS0CXgIuWdG5\ntGsj23ZaIK4wnErDAMxshpndG6dsjuMU7xCmosYSppfeB14EVpf0CfDfwKi08v2AcZIeM7NPo8/f\niKZ57siz7/rO3w5cIOk9YL16yjTU7g+S3idkqjsnOn894Xt9JOlj4M852nacOnh4c8dxHCcWPsJw\nHMdxYuEKw3Ecx4mFKwzHcRwnFq4wHMdxnFi4wnAcx3Fi4QrDcRzHiYUrDMdxHCcW/w9Aaiblpksq\n0AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa0cc198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure(1)\n",
    "plt.subplot(211)\n",
    "plt.plot(plotdata[\"batchsize\"], plotdata[\"loss\"], 'b--')\n",
    "plt.xlabel('Minibatch number')\n",
    "plt.ylabel('Loss')\n",
    "plt.title('Minibatch run vs. Training loss ')\n",
    "plt.show()\n",
    "\n",
    "plt.subplot(212)\n",
    "plt.plot(plotdata[\"batchsize\"], plotdata[\"error\"], 'r--')\n",
    "plt.xlabel('Minibatch number')\n",
    "plt.ylabel('Label Prediction Error')\n",
    "plt.title('Minibatch run vs. Label Prediction Error ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the trend for the label prediction error is still close to 50%. Remember that this is time variant, therefore it is expected that the system will have some noise as it trains through time. It should be noted; the model is still learning the market. Additionally, since this time series data is so noisy, having an error rate below 50% is good (many trading firms have win-rates of near 50% and have made money nearly every day [VIRTU](https://en.wikipedia.org/wiki/Virtu_Financial#Trading_activity)). However note they are high frequency trading firm and can leverage themselves up with low winrate strategies (51%). Trying to classify and trade every single day is expensive from transaction fees perspective. Therefore, one approach would be to trade when we think we are more likely to win?\n",
    "\n",
    "Let us try this idea."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average error: 47.48%\n"
     ]
    }
   ],
   "source": [
    "# Now that we have trained the net, and we will do out of sample test to see how we did.\n",
    "# and then more importantly analyze how that set did\n",
    "\n",
    "test_features = np.ascontiguousarray(test_data[predictor_names], dtype = \"float32\")\n",
    "test_labels = np.ascontiguousarray(test_data[[\"next_day\",\"next_day_opposite\"]], dtype=\"float32\")\n",
    "\n",
    "avg_error = trainer.test_minibatch({input : test_features, label : test_labels})\n",
    "print(\"Average error: {0:2.2f}%\".format(avg_error * 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we see that we have an error rate near 50%. At first glance this may appear to not have learned the network, but let us examine further and see if we have some predictive power."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "out = C.softmax(z)\n",
    "predicted_label_prob = out.eval({input:test_features})\n",
    "test_data[\"p_up\"] = pd.Series(predicted_label_prob[:,0,0], index = test_data.index)\n",
    "test_data[\"p_down\"] = predicted_label_prob[:,0,1]\n",
    "test_data['long_entries'] = np.where((test_data.p_up > 0.55), 1, 0)\n",
    "test_data['short_entries'] = np.where((test_data.p_down > 0.55) , -1, 0)\n",
    "test_data['positions'] = test_data['long_entries'].fillna(0) + test_data['short_entries'].fillna(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating our Data\n",
    "Here we take the output of our test set and compute the probabilities from the softmax function. Since we have probabilities we want to trade when there is a \"higher\" chance that we will be right, instead of just a >50% chance that the market will go in one direction. The goal is to find a signal, instead of trying to classify the market. Since the market is so noisy we want to only trade when we have an \"edge\" on the market. Moreover, trading frequently has higher fees (you have to pay each time you trade).\n",
    "\n",
    "We will say that if the prediction probability is greater than 55% (in either direction) we will take a position in the market. If it shows that the market will be up the next day with greater than 55% probability, we will take a 1-day long. If it is greater than a 55% chance that the next day will be below today's position we will take 1-day [short] (http://www.investopedia.com/university/shortselling/shortselling1.asp)(the same as borrowing a stock and buying it back). \n",
    "\n",
    "We will then evaluate this timeseries performance by looking at some more metrics: average monthly return, standard deviation of monthly returns, the [Sharpe ratio](http://www.investopedia.com/terms/s/sharperatio.asp), and the [Maximum drawdown](https://en.wikipedia.org/wiki/Drawdown_(economics)). The Sharpe ratio is the average return minus the risk free rate (which is basically zero) over the standard deviation of returns normalized to a year. \n",
    "\n",
    "$$Sharpe = \\frac{r_p - r_f}{\\sigma_p}$$\n",
    "$$r_p = \\text{portfolio return}$$\n",
    "$$r_f = \\text{risk free rate}$$\n",
    "$$\\sigma_p = \\text{standard deviation of portfolio return}$$\n",
    "\n",
    "Generally, the higher the Sharpe ratio, the better you are taking less risk for each unit of reward. This assumes the mean return and standard deviation are sufficient to describe the distribution of returns, akin to an assumption of normally distributed returns.\n",
    "\n",
    "A trading strategy can be profitable even if the winrate is 50% or lower, if the wins are greater than the losses you can have a less than 50% winrate and still make some money (usually classified as momentum strategies). Finally, even if we do not beat the market by trading it individually, we can still outperform it by trading multiple assets that are uncorrelated with each other (or trade a basket of stocks to get market exposure; we are not trading each at the same time). However, that will not be covered in this tutorial. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def create_drawdowns(equity_curve):\n",
    "    \"\"\"\n",
    "    Calculate the largest peak-to-trough drawdown of the PnL curve\n",
    "    as well as the duration of the drawdown. Requires that the \n",
    "    pnl_returns is a pandas Series.\n",
    "\n",
    "    Parameters:\n",
    "    pnl - A pandas Series representing period percentage returns.\n",
    "\n",
    "    Returns:\n",
    "    drawdown, duration - Highest peak-to-trough drawdown and duration.\n",
    "    \"\"\"\n",
    "\n",
    "    # Calculate the cumulative returns curve \n",
    "    # and set up the High Water Mark\n",
    "    # Then create the drawdown and duration series\n",
    "    hwm = [0]\n",
    "    eq_idx = equity_curve.index\n",
    "    drawdown = pd.Series(index = eq_idx)\n",
    "    duration = pd.Series(index = eq_idx)\n",
    "\n",
    "    # Loop over the index range\n",
    "    for t in range(1, len(eq_idx)):\n",
    "        cur_hwm = max(hwm[t-1], equity_curve[t])\n",
    "        hwm.append(cur_hwm)\n",
    "        drawdown[t]= (hwm[t] - equity_curve[t]) \n",
    "        duration[t]= 0 if drawdown[t] == 0 else duration[t-1] + 1\n",
    "    return drawdown.max(), duration.max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEZCAYAAABmTgnDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucHGWZ9//PNxxMAoQhQYITYCYcdgg8SERg9eGQcBBB\nNLA8K2AUM8Aui2TRXXUhcdTA/pgNsKuyaljNLpIBiRB05eAiJAGGw4rESJBDwhgjaULGjJBIIIZD\nQq7fH1U9dIaZ6erprunqe6736zWvVFXfXX1d05O+uu676i6ZGc4551x/hlU7AOecc9nnxcI551xR\nXiycc84V5cXCOedcUV4snHPOFeXFwjnnXFFeLFzJJP2HpJYK7WtfSa9KUrz+oKQLKrHveH/3SDqv\nUvsr4XWvkvSSpM5BeK3nJZ04wOf2+fvu772RNFXSvf3s91hJKwYSk8smLxZuO5JWS9osaaOkDZIe\nlfR3+Q8MADP7nJm1JthX0Q8xM1tjZqOsAhf8SJol6aYe+/+Ymd1c7r5LjGNf4IvAwWZW38vjkyS9\nHX8Qb5S0QlLzYMaYRH/vjZnNN7NT8+uStknav+DxR81swmDF6tLnxcL1ZMDpZrY70ABcDVwO3FDp\nF5K0Q6X3mRENwMtmtr6fNmvjD+LdgRnAf0o6uGejGvod+dW9gfNi4XojADN7zcx+BpwDTJN0CICk\nGyX9c7w8RtLdkv4kab2kh+LtNwH7AXfH36C/LKkh/gZ6gaQccH/BtsK/xQMlPR5/6/6ppLp4n5Mk\nrdku0PjoRdJHga8A50h6TdKy+PHCrhNJ+mp89LRO0jxJo+LH8nF8VlJO0h8lfaXPX5A0StJNcbvn\n891ykk4CFgL1cd4/KPbLNrM7gT8Bh/T2O4r3O0XSM/HR3gO9FJajJT0bvwc3SNo5fl5d/P78MX7s\nbknjejy3r993b+9NPv9pkh6Jlx8i+pt5Ks75kz3fK0nvk/TjOI5Vki4teOwoSb+KX/8Pkv6t2O/M\nDT4vFq4oM/sV8CJwXC8PfwlYA4wB9iL6wMbMPgu8AHw8/gZd+AFwPHAw8NH8S/TY53lAM7A38Dbw\nncJw+ojxPuBfgNvMbDcz+0Avzc4HPgtMAvYHdgO+26PNMcBBwMnA1yU19fZ68fN2AxqBycBnJZ1v\nZvcDpwGdcd79jr/EBeyvgN2Bpwse6v4dSToImA98Hngv8HOiIrxjQfupwEeAA4Am4Kvx9mHAD4B9\niYr35l5yLvn3XfiYmU2K1w+Lc7698PG4C/NuYBnwPuAk4AuSPhK3+3fguvgo6wBgQT+v6arEi4VL\nqhMY3cv2LUQfAOPN7G0z+98ej6vHugGzzOx1M3uzj9e62cxWmNnrwNeATxaOmZRhKvBNM8uZ2WZg\nJnBuwTdnA64ws7fM7CngN8DhPXcStz8HmGFmm80sB3yD6EM3qXGSNgAvEeX4GTNbWRBH4e/oHOBn\nZvaAmb0N/BswAvi/Bfv7jpl1mtkrQCvwKQAz22BmPzWzN83sz8BsokJUqOfv++wB/r77es7RwJ5m\n1hr/jawG/gs4N358C9HRzZj497lkAK/tUubFwiU1DtjQy/Z/BVYBCyX9TtLlCfb1YpHHC7uacsBO\nwJ6Jouxffby/wn3vCIwt2NZVsLwZ2LWX/ewZP++FHvvq2b3Tn7VmNtrM9jSzIwq+jecV/o62izse\ncF7T4/UK2+fi5yBphKTvx11vrwAPAXU9ikFav++8/YiLY/zzJ6JCvVf8+AVER0PPxd1hp1fwtV2F\neLFwRUk6iujD55Gej5nZJjP7spkdAEwBvijphPzDfeyy2GDovgXLDUTfPF8G/gyMLIhrB6JumaT7\n7Yz313PfXb0379PL8fN67mttifvpT2EuPeOG6Hf0Yo/1wljyp+x+mahb7Sgzq+Odowr189y3iHKs\nlDXA7+PiONrM9jCz3c3sEwBmtsrMpprZe4FrgR9LGlHB13cV4MXC9UnSbpI+DvyIqKtieS9tTpd0\nQLz6GrCVqN8bog/h/Xs+pbeX6rH+GUkHSxoJXAncHn+b/i0wXNJpcX/9V4GdC57XBTT204XyI+Af\nJTVK2pWou+ZWM9vWT2zvErdfALRK2lVSA/CPQKVO0e0ZxwLgdEknSNpR0peBN4DHCtpMlzRO0mii\ncaNb4+27Aq8Dr8aPXdHL6/X1++4tlr6s493vdd4S4DVJl0kaLmkHSYdKOhJA0qcl5Y9kNhIVym19\n7MtViRcL15u7JW0k6maZSdRH3tdA7UHAYkmvAf8LzDGzh+PHZgNfi7sevhhv6+3bv/VYvhloI/p2\nvDPwBQAzexW4hOg03heJilPht+vbiT7c1kta2su+fxDv+2GirrPNRIPGvcXRV6x5n4+f//t4fz80\nsxv7aV+K7V7XzH4LfIZoYPol4HTgE2a2taD9fKKzsH4HrCQqhADXER2NvQz8Arinl9fq9ffdSyz9\n/T6uAG6K3+u/7hH/NuDjwETgeeCPwH8Co+ImpwLPSnoV+BZwTj/jWa5KlPbNjyR9AfibePU/zezb\nkvYAbiM65F0NnG1mG+P2M4k+mLYCXzCzhakG6JxzrqhUjywkHQpcCBxJ9K3i43GXxQxgsZk1AQ8Q\nfXtF0Xn8ZwMTiE4/vL5CZ8E455wrQ9rdUBOAx+PT9t4mOlw/i2ggtC1u0wacGS9PIepD3hqfXreS\n6LQ755xzVZR2sXgGOE7SHvHg2ceIzrwYa2ZdAGa2jndOoRvH9qfxraW00xGdc86lYMfiTQbOzJ6T\ndA2wCNhEdAXn2701TTMO55xz5Um1WADEZ4jcCCCplejIoUvSWDPrkrQ30dkREB1JFJ7zvQ+9nLsu\nyYuLc84NgJkNaBw49VNnJb03/nc/4K+ITvG7i2guGoBpwJ3x8l1E0y/sLGk8cCDROdrvYmbB/sya\nNavqMXh+nt9QzC/k3MzK+46d+pEF8JP4YqAtwCVm9mrcNbVA0WygOaIzoDCz5ZIWAMsL2g+5o4jV\nq1dXO4RUeX61LeT8Qs6tXIPRDdVz0jLMbAPRrJ69tZ9NdDGXc865jPAruDOoubm52iGkyvOrbSHn\nF3Ju5Ur9Cu40SBqKvVPOOVcWSVhWB7hd6drb26sdQqo8v9pWbn6NjY1I8p8UfxobGyvyXhcajAFu\n55zrlsvlyj4zx/UvjVmSvBvKOTeo4q6QaocRtL5+x94N5ZxzLlXeDZVB7e3tTJ48udphpGYw8ptz\nwxw613cWbxirH1PP9AunV+S1K5nfnDnz6ezclKhtff2uTJ8+tSKv25/Q/z5d77xYuCB1ru+k4aSe\ndyLtW+7+XPFGVdDZuYmGhosStc3l5qYcjRvKvFhkUOjf2jy/2pZGfqUcQQ3EYB11hcyLhXOu6ko5\nghoIP+oqnw9wZ5Cfp1/bPL/aNX78eK6++moOPfRQxowZw4UXXshbb73VZ/u2tjaOO+647bYNGzaM\n3//+9wCcf/75fO5zn+OUU05h1KhRnHDCCbzwwgup5pAWLxbOOVdg/vz5LFq0iFWrVtHR0cFVV13V\nb/ue1zT0XJ8/fz6zZs1i/fr1HH744Xz605+ueMyDwYtFBnmfd23z/GrbpZdeSn19PXV1dbS0tPCj\nH/2opOf3vL7h9NNP55hjjmGnnXaitbWVxx57jLVr33WbnszzYuGccwX22Wef7uWGhgY6O5Ofgt2b\nffd9535uu+yyC6NHjy57n9XgxSKDQu4TBs+v1oWe35o1a7qXc7kc9fX1fbbdZZdd2Lx5c/f6unXr\n+t3fpk2b2LBhQ7/7zCovFs45V2DOnDmsXbuWDRs28C//8i+ce+65fbY9/PDDefbZZ3nqqad48803\nufLKK981ZnHPPffwi1/8grfeeouvfe1rfPjDH2bcuHFpp1FxqZ86K+kfgQuBbcDTwPnALsBtQAOw\nGjjbzDbG7WcCFwBbgS+Y2cK0Y8ya0PuEPb/alkZ+9fW7pnp6a339ronbTp06lVNOOYU//OEPnHnm\nmbS0tPTZ9qCDDuLrX/86J510EiNHjmT27NnMnbt9HlOnTuWKK67gscce44Mf/CA//OEPB5xHNaVa\nLCTVA5cCB5vZW5JuAz4FHAIsNrNrJV0OzARmSDqE6BarE4B9gMWSDvJZA50LW5YumDvqqKO4/PLL\nE7efOXMmM2fO7F6fOnX7XPbcc0+uv/76isVXLYPRDbUDsIukHYERwFrgDKAtfrwNODNengLcamZb\nzWw1sBI4ehBizJTQ+4Q9v9oWen6ud6kWCzPrBL4BvEBUJDaa2WJgrJl1xW3WAXvFTxkHrCnYxdp4\nm3POpa63+0DMnj2b3XbbjVGjRm33c/rppw9of7Uq7W6oOqKjiAZgI3C7pE8DPbuVSu5mam5u7r4b\nVF1dHRMnTuzuS81/86nV9fy2rMRTi/nlVuW6JxLsWNoBQNORTX2ud63q2i62LOXX0RGtNzX1vz58\neGXiTzu/LMtfeV2oZzdTKX7wgx+UG1JZ2tvbmTdvHkDZd89L9eZHkv4a+KiZ/W28fh7wIeBEYLKZ\ndUnaG3jQzCZImgGYmV0Tt78XmGVmj/fYrw9juH61XNtS8qyzrZe1phjRwLS0zC1p1tnW1vTmV6oU\nv/lR+mrx5kcvAB+SNFzR8dhJwHLgLqA5bjMNuDNevgs4V9LOksYDBwJLUo4xc0LvE/b8alvo+bne\npdoNZWZLJP0YWAZsif+dC+wGLJB0AZAjOgMKM1suaQFRQdkCXOKHEM45V32pX2dhZlcCV/bYvAE4\nuY/2s4HZaceVZbXQt1sOz6+2hZ6f651fwe2cc64oLxYZFHqfsOdX20LPr1yF97MIid8pzzlXdXNu\nmEPn+vRmYq0fU8/0C6entv9CIV1bUciLRQaF3ifs+dW2NPLrXN9Z0qnOpcrdn0tt3z2Fek6Od0M5\n51ys1NuqAvzrv/4r9fX17LPPPtx4443bHVm8+uqrfPazn2WvvfZi/PjxtLa+cy1PY2Mjy5YtA+CW\nW25h2LBhrFixAogu5jvrrLMAuPLKKznnnHOYNm0ao0aN4rDDDuOJJ56odOpFebHIoND7hD2/2hZ6\nfqXcVvXee+/lm9/8Jvfffz8rV65k8eLF2z3+93//97z22musXr2a9vZ2brrpJm688UYAJk2a1P27\nfPjhhznggAN4+OGHAXjooYeYNGlS937uvvtupk6dysaNG/nEJz7B9OmD06VWyIuFc84VKOW2qrff\nfjvnn38+EyZMYMSIEVxxxRXd3VDbtm3jtttu4+qrr2bkyJE0NDTwpS99iZtvvhmIisVDDz0EwCOP\nPMLMmTO713sWi2OPPZaPfvSjSOK8887jqaeeSiv9PnmxyCDv865tnl9tK+W2qp2dndvdNrWh4Z1x\nl5dffpmtW7ey3377bfd4/v7bkyZN4pFHHmHdunVs27aNs88+m0cffZRcLserr77KxIkTu5+39957\ndy+PHDmSN954g23btpWXaIm8WDjnXIFSbqv6vve9713t82MWe+65JzvttBO5XG67x/N3yTvggAMY\nMWIE3/nOdzj++OPZdddd2XvvvZk7dy7HHntspdMqmxeLDAq9T9jzq22h51fKbVXPPvts5s2bx4oV\nK9i8eTP//M//3P3YsGHDOPvss2lpaWHTpk3kcjm+9a1vcd5553W3mTRpEt/97ne7u5wmT5683Xpf\nqnHGlZ8665yruvox9ame3lo/pu+jg55Kua3qqaeeyj/8wz9w4oknssMOO3DVVVcxf/787se//e1v\nc+mll7L//vszYsQILrroIs4///zuxydNmsStt97K8ccf373+jW98o3u9L9W4liPVKcrT4lOUu2J8\nivLsyvIU5ePHj+eGG27gxBNPrHYoZanFKcqdc84FwItFBoXeJ+z51baQ86v0bVVD4mMWzjkXq/Rt\nVUPiRxYZFPp57J5fbQs9P9e7VIuFpL+QtEzSE/G/GyV9XtIekhZK6pB0n6TdC54zU9JKSSsknZJm\nfM4555JJtViY2W/N7ANmdgTwQeDPwE+BGcBiM2sCHgBmAkg6hOgWqxOA04DrFep8v/0IuU8YPL9a\nF3p+rneDOWZxMrDKzNZIOgPIX3XSBrQTFZApwK1mthVYLWklcDTw+CDG6ZxLUUNDQ7D3fMiKwmlH\nKmUwi8U5QP5qlbFm1gVgZusk7RVvHwc8VvCctfG2ISX0PmHPr7aVm9/q1asrEocbXINSLCTtRHTU\ncHm8qefVIiVfodPc3ExjYyMAdXV1TJw4sfuPOH+Y7OtDdz23Ktd9UV7H0g4Amo5s6nO9a1UXeVmI\nv3C9oyNab2rqf3348GzG7+vVW29vb2fevHkA3Z+XAzUoV3BLmgJcYmanxusrgMlm1iVpb+BBM5sg\naQZgZnZN3O5eYJaZPd5jf0Ffwd3e3t79xodoMPKr5hXclcwvi1dwh/z3GXJuUBtXcH8KKJwU/i6g\nOV6eBtxZsP1cSTtLGg8cCCwZpBidc871IfVuKEkjiQa3C7/yXAMskHQBkCM6AwozWy5pAbAc2EJ0\nNBLuIUQfQv5mA55frQs5v5BzK1fqxcLMNgPv7bFtA1EB6a39bGB22nE555xLzq/gzqD8AFWoPL/a\nFnJ+IedWLi8WzjnnivJikUGh95t6frUt5PxCzq1cXiycc84V5cUig0LvN/X8alvI+YWcW7m8WDjn\nnCvKi0UGhd5v6vnVtpDzCzm3cnmxcM45V5QXiwwKvd/U86ttIecXcm7l8mLhnHOuKC8WGRR6v6nn\nV9tCzi/k3MrlxcI551xRXiwyKPR+U8+vtoWcX8i5lcuLhXPOuaK8WGRQ6P2mnl9tCzm/kHMrlxcL\n55xzRaVeLCTtLul2SSskPSvpLyXtIWmhpA5J90navaD9TEkr4/anpB1fFoXeb+r51baQ8ws5t3IN\nxpHFvwP3mNkE4HDgOWAGsNjMmoAHgJkAkg4husXqBOA04HpJA7q5uHPOucpJtVhIGgUcZ2Y3ApjZ\nVjPbCJwBtMXN2oAz4+UpwK1xu9XASuDoNGPMotD7TT2/2hZyfiHnVq60jyzGAy9LulHSE5LmShoJ\njDWzLgAzWwfsFbcfB6wpeP7aeJtzzrkq2nEQ9n8EMN3Mlkr6FlEXlPVo13O9qObmZhobGwGoq6tj\n4sSJ3d8K8v2Otbp+3XXXBZVPNfLLrcrRcFIDAB1LOwBoOrKpz/WuVV3kZS2/jo5ovamp//XhwysT\nfxbev2qtF45ZZCGeSuQzb948gO7Py4GSWcmf08l3Lo0FHjOz/eP1Y4mKxQHAZDPrkrQ38KCZTZA0\nAzAzuyZufy8wy8we77FfSzPuamtvb+9+40M0GPm1XNvSXSySyN2fo/Wy1oq8diXza2mZS0PDRYna\n5nJzaW1N1rYcIf99hpwbgCTMbEDjwKl2Q8VdTWsk/UW86STgWeAuoDneNg24M16+CzhX0s6SxgMH\nAkvSjDGLQv5jBc+v1oWcX8i5lSvtbiiAzwO3SNoJ+D1wPrADsEDSBUCO6AwozGy5pAXAcmALcEnQ\nhxDOOVcjUj911sx+Y2ZHmdlEMzvLzDaa2QYzO9nMmszsFDN7paD9bDM70MwmmNnCtOPLosJ+0xB5\nfrUt5PxCzq1cfgW3c865orxYZFDo/aaeX20LOb+QcyuXFwvnnHNFebHIoND7TT2/2hZyfiHnVi4v\nFs4554ryYpFBofeben61LeT8Qs6tXF4snHPOFeXFIoNC7zf1/GpbyPmFnFu5EhULSYelHYhzzrns\nSnpkcb2kJZIuKbyrnUtH6P2mnl9tCzm/kHMrV6JiYWbHAZ8G9gV+LWm+pI+kGplzzrnMSDxmYWYr\nga8ClwOTgG9Lek7SWWkFN1SF3m/q+dW2kPMLObdyJR2zeH9846IVwInAJ+J7ap8IfCvF+JxzzmVA\n0inKvwP8F/AVM3s9v9HMOiV9NZXIhrDQ+009v9oWcn4h51aupMXidOB1M3sbQNIwYLiZbTazm1OL\nzjnnXCYkHbNYDIwoWB8ZbytK0mpJv5G0TNKSeNsekhZK6pB0X+EZVpJmSlopaYWkU5ImEpLQ+009\nv9oWcn4h51aupMViuJltyq/EyyMTPncb0f22P2BmR8fbZgCLzawJeACYCSDpEKK75k0ATiM6ZXdA\n94t1zjlXOUmLxZ8lHZFfkfRB4PV+2hdSL69zBtAWL7cBZ8bLU4BbzWyrma0GVgJHM8SE3m/q+dW2\nkPMLObdyJR2z+AfgdkmdRB/+ewPnJHyuAYskvQ1838z+CxhrZl0AZrZO0l5x23HAYwXPXRtvc845\nV0VJL8r7FXAw8DngYmCCmf064WscY2ZHAB8Dpks6jqiAbPcSCfc1JITeb+r51baQ8ws5t3IlPbIA\nOApojJ9zhCTM7KZiTzKzP8T/viTpDqJupS5JY82sS9LewB/j5muJrhLP2yfe9i7Nzc00NjYCUFdX\nx8SJE7sPIfNveK2uP/nkk5mKpxbzy63K0XBSAwAdSzsAaDqyqc/1rlVd5GUtv46OaL2pqf/14cMr\nE38W3j9fr8x6e3s78+bNA+j+vBwomRX/Ui/pZuAA4Eng7XizmdnnizxvJDDMzDZJ2gVYCFwJnARs\nMLNrJF0O7GFmM+IB7luAvyTqfloEHGQ9gpTUc5Nz22m5tqW7WCSRuz9H62WtKUY0MC0tc2louChR\n21xuLq2tydq6oSn+kj+gk4aSHlkcCRwygE/oscBPJVn8WreY2UJJS4EFki4AckRnQGFmyyUtAJYD\nW4BLvCo451z1JT0b6hmiQe2SmNnzZjYxPm32MDO7Ot6+wcxONrMmMzvFzF4peM5sMzvQzCaY2cJS\nXzME+cPIUHl+tS3k/ELOrVxJjyz2BJbHF9W9md9oZlNSico551ymJC0WV6QZhNtefqAqVJ5fbQs5\nv5BzK1eiYmFmD0lqIBpsXhwPXO+QbmjOOeeyIukU5X8L/Bj4frxpHHBHWkENdaH3m3p+tS3k/ELO\nrVxJB7inA8cAr0L3jZD26vcZzjnngpG0WLxpZm/lVyTtiF91nZrQ+009v9oWcn4h51aupMXiIUlf\nAUbE996+Hbg7vbCcc85lSdJiMQN4CXga+DvgHqL7cbsUhN5v6vnVtpDzCzm3ciU9G2ob8J/xj3PO\nuSEmUbGQ9Dy9jFGY2f4Vj8gF32+axfyWLltKy7UtidvXj6ln+oXTe32sv/zmzJlPZ+emPh9/V1xL\nn6Eh+RRXgyKL71+lhJxbuUqZGypvOPBJYHTlw3GuOja9uankiQcHorNzU+KJAQHa2y8e0Os4V2lJ\n72exvuBnrZldB5yecmxDVuj9pp5fbQs5v5BzK1fSbqgjClaHER1plHIvDOecczUs6Qf+NwqWtwKr\niacVd5UXer+p51fbQs4v5NzKlfRsqBPSDsQ551x2JZ0b6ov9/aQd5FATer+p51fbQs4v5NzKlfSi\nvCOBzxFNIDgOuBg4Atgt/umXpGGSnpB0V7y+h6SFkjok3Sdp94K2MyWtlLRC0imlJuScc67yko5Z\n7AMcYWavAUi6AvgfM/tMwud/gehWqaPi9RnAYjO7Nr4H90wgfw/us4EJ8WsulvSue3CHLvR+U8+v\ntoWcX8i5lSvpkcVY4K2C9bfibUVJ2gf4GPBfBZvPANri5TbgzHh5CnCrmW01s9XASuDohDE655xL\nSdJicROwRNIV8VHF47zzYV/Mt4B/YvsrwMeaWReAma3jnenOxwFrCtqtjbcNKaH3m3p+tS3k/ELO\nrVxJz4ZqlfRz4Lh40/lmtqzY8ySdDnSZ2ZOSJvf3EknicM45Vx2lXFg3EnjVzG6U9F5J483s+SLP\nOQaYIuljwAhgN0k3A+skjTWzLkl7A3+M268F9i14/j7xtndpbm6msbERgLq6OiZOnNjd35j/dlCr\n6/ltWYmnFvPLrcp1T9/RsbQDgKYjm/pc3/jSxu7YkrTvWtW1XS6l5NfREa03NVV2ffjw3uOpxfev\nWuuTJ0/OVDzlrre3tzNv3jyA7s/LgVKSsWNJs4jOiGoys7+QVA/cbmbHJH4haRLwJTObIulaYL2Z\nXRMPcO9hZvkB7luAvyTqflpEdN9v67GvoTbm7UrUcm1LSXM9tbW2Ma1lWuL2uftztF7WWnpcLXNL\nmhuqre1ipk37XrKYcnNpbU2+bzf0SMLMNJDnJh2z+Cuiwec/A5hZJwlOme3H1cBHJHUAJ8XrmNly\nYAHRmVP3AJcMxaqQ/2YQKs+vtoWcX8i5lStpN9RbZmaSDEDSLqW+kJk9BDwUL28ATu6j3Wxgdqn7\nd845l56kRxYLJH0fqJP0t8Bi/EZIqSnsGw6R51fbQs4v5NzKlfRsqH+L7739KtAEfN3MFqUamXPO\nucwoemQhaQdJD5rZIjP7JzP7sheKdIXeb+r51baQ8ws5t3IVLRZm9jawrXD+Juecc0NL0gHuTcDT\nkhYRnxEFYGafTyWqIS70flPPr7aFnF/IuZUrabH47/jHOefcENRvN5Sk/QDMrK23n8EJcegJvd/U\n86ttIecXcm7lKjZmcUd+QdJPUo7FOedcRhUrFoWXhe+fZiDuHaH3m3p+tS3k/ELOrVzFioX1seyc\nc24IKVYsDpf0qqTXgPfHy69Kek3Sq4MR4FAUer+p51fbQs4v5NzK1e/ZUGa2w2AF4pxzLruSzg3l\nBlHo/aaeX20LOb+QcyuXFwvnnHNFebHIoND7TT2/2hZyfiHnVi4vFs4554pKtVhIeo+kxyUtk/R0\nfHtWJO0haaGkDkn3FU5SKGmmpJWSVkg6Jc34sir0flPPr7aFnF/IuZUr1WJhZm8CJ5jZB4CJwGmS\njgZmAIvNrAl4AJgJEN+D+2xgAnAacL2kAd0v1jnnXOWk3g1lZpvjxfcQnaprwBlAfm6pNuDMeHkK\ncKuZbTWz1cBK4Oi0Y8ya0PtNPb/aFnJ+IedWrtSLhaRhkpYB64BFZvYrYKyZdQGY2Tpgr7j5OGBN\nwdPXxtucc85VUdIpygfMzLYBH5A0CvippEN599QhJU8l0tzcTGNjIwB1dXVMnDixu78x/+2gVtfz\n27ISTy3ml1uVo+GkBgA6lnYA0HRkU5/rG1/a2B1bkvZdq7q2y6WU/Do6ovWmpsquDx/eezy1+P5V\na33y5MnbY+FDAAARq0lEQVSZiqfc9fb2dubNmwfQ/Xk5UDIbvCmfJH0N2Az8DTDZzLok7Q08aGYT\nJM0AzMyuidvfC8wys8d77McGM25Xe1qubekuFkm0tbYxrWVa4va5+3O0XtZaelwtc2louCh5XG0X\nM23a95LFlJtLa2vyfbuhRxJmNqBx4LTPhtozf6aTpBHAR4AVwF1Ac9xsGnBnvHwXcK6knSWNBw4E\nlqQZYxblvxmEyvOrbSHnF3Ju5Uq7G+p9QJukYUSF6TYzu0fSL4EFki4AckRnQGFmyyUtAJYDW4BL\n/BDCOeeqL9ViYWZPA0f0sn0DcHIfz5kNzE4zrqwr7BsOkedX20LOL+TcyuVXcDvnnCvKi0UGhd5v\n6vnVtpDzCzm3cnmxcM45V5QXiwwKvd/U86ttIecXcm7l8mLhnHOuKC8WGRR6v6nnV9tCzi/k3Mrl\nxcI551xRXiwyKPR+U8+vtoWcX8i5lcuLhXPOuaK8WGRQ6P2mnl9tCzm/kHMrlxcL55xzRXmxyKDQ\n+009v9oWcn4h51YuLxbOOeeK8mKRQaH3m3p+tS3k/ELOrVxeLJxzzhXlxSKDQu839fxqW8j5hZxb\nudK+reo+kh6Q9KykpyV9Pt6+h6SFkjok3Ze/9Wr82ExJKyWtkHRKmvE555xLJu0ji63AF83sUODD\nwHRJBwMzgMVm1gQ8AMwEkHQI0S1WJwCnAddLGtDNxWtZ6P2mnl9tCzm/kHMrV6rFwszWmdmT8fIm\nYAWwD3AG0BY3awPOjJenALea2VYzWw2sBI5OM0bnnHPFDdqYhaRGYCLwS2CsmXVBVFCAveJm44A1\nBU9bG28bUkLvN/X8alvI+YWcW7l2HIwXkbQr8GPgC2a2SZL1aNJzvajm5mYaGxsBqKurY+LEid1v\ndP5Q0teH7npuVY6GkxoA6FjaAUDTkU19rm98aSN5Sdp3rerqbl9qfB0d0XpTU2XXhw8fWDy+Hu56\ne3s78+bNA+j+vBwomZX8OV3aC0g7Aj8Dfm5m/x5vWwFMNrMuSXsDD5rZBEkzADOza+J29wKzzOzx\nHvu0tOOupvb29u43PkSDkV/LtS3dxSKJttY2prVMS9w+d3+O1stae32sv/xaWubS0HBR8rjaLmba\ntO8liyk3l9bW5PseqJD/PkPODUASZjagceDBOLL4AbA8XyhidwHNwDXANODOgu23SPoWUffTgcCS\nQYjRuZIsXbaUlmtbWPrrZ9n02lvbPbZx/Xp2HzOme33XnXfnyENPiJ639Bkaktcw5zIj1WIh6Rjg\n08DTkpYRdTd9hahILJB0AZAjOgMKM1suaQGwHNgCXBL0IUQfQv5mA2Hkt+nNTTSc1MCy13KMH3N8\nv203PJ7rPppob794MMJLVQjvX19Czq1cqRYLM/tfYIc+Hj65j+fMBmanFpRzzrmSDcoAtytN6P2m\nA81vzg1z6Fzfmajt0t8sLWnMotCD7UvY+Mob/bbJrV7HHXc8TC7Xyegx2z/W9bsOxh7Y9E7bNUu5\n48GWaPmVJd3Lfdl9RD0nfGh6yXEvXfokLS1zE7evr9+V6dOnlvw6If99hpxbubxYuJrRub4zcQFo\n/2X7gF9n4ytvMLpI19J7hj/P6DHH09Hxw6L7e2PbJkb/ZRT3e97etXu5LxsezyUPtsCmTdtKGjzP\n5ZIXFud8bqgMCv2bTej5FR5VhCjk9y/k3MrlxcI551xRXiwyKH9RTahCz6/rdx3VDiFVIb9/IedW\nLi8WzjnnivJikUGh95uGnp+PWdSukHMrlxcL55xzRXmxyKDQ+01Dz8/HLGpXyLmVy4uFc865orxY\nZFDo/aah5+djFrUr5NzK5cXCOedcUT7dRwaFPj/NYOdXylxPQK/zPZWi59xQpSplLqmBziNVjpD/\nPkPOrVxeLFzwSpnrCUg031OaSplLaqDzSDlXKu+GyqDQv9mEnp+PWdSukHMrlxcL55xzRaVaLCTd\nIKlL0lMF2/aQtFBSh6T7JO1e8NhMSSslrZB0SpqxZVno53qHnp9fZ1G7Qs6tXGkfWdwIfLTHthnA\nYjNrAh4AZgJIOoTo9qoTgNOA6yUN6MbizjnnKivVYmFmjwJ/6rH5DKAtXm4DzoyXpwC3mtlWM1sN\nrASOTjO+rAq93zT0/HzMonaFnFu5qjFmsZeZdQGY2Tpgr3j7OGBNQbu18TbnnHNVloVTZ20gT2pu\nbqaxsRGAuro6Jk6c2P2tIN/vWKvr1113XVD5VCq/vI6l0ZhA05FNfa5vfGljd/v1q7vY8qd3rn3I\njykUrr++8Z32r//xle2ulSjWvufjKx5azOhx+3avv75xY9H9Fa6X2r6jo8fvJ15vaprc7/rw4VF7\n//t8Z73wby0L8VQin3nz5gF0f14OlMwG9Fmd/AWkBuBuM3t/vL4CmGxmXZL2Bh40swmSZgBmZtfE\n7e4FZpnZ473s09KOu5pCvzBooPm1XNuS+B7cba1tTGuZBsAddzxc9DqLxxa08eGzo/aP/eKHfPj/\nfiZR+97a9rwob6D7TtJ+w+M5zjyhFYC2touZNu17/e67UC43l9bW5Pfszgv57zPk3AAkYWYDGgse\njG4oxT95dwHN8fI04M6C7edK2lnSeOBAYMkgxJc5If+xQvj5+ZhF7Qo5t3Kl2g0laT4wGRgj6QVg\nFnA1cLukC4Ac0RlQmNlySQuA5cAW4JKgDx+cc66GpH021FQzqzez95jZfmZ2o5n9ycxONrMmMzvF\nzF4paD/bzA40swlmtjDN2LIs9HO9Q8/Pr7OoXSHnVi6/gts551xRXiwyKPR+09Dz8zGL2hVybuXK\nwqmzzpVksKccz7JSpjOH6kxp7sLgxSKDQj99r9z8sj7leLn3syhFKdOZQ2WmNA/57zPk3Mrl3VDO\nOeeK8mKRQaF/swk9Px+zqF0h51YuLxbOOeeK8jGLDEqr33Tbtm08//zzJL3WcdiwYTQ2NjJsWGW/\nU4TeLzyYYxbVEPL7F3Ju5fJiMYS88MILfP+O77Nz3c6J2m95ZQvTPzmd/fbbL+XI3FA3Z858Ojs3\nJWpbX78r06dPTTki15MXiwxK65uNmfGeUe9h34n7Jmq/5tdrEh+FlCKf35wb5tC5vjPx85b+Zmni\niQSrqVaOKpYufZKWlrmJ2hZ+QKfx99nZuYmGhmSTGuZyyWIeCD+q6JsXC1c1nes7S/rwb/9le3rB\nDEGbNm3LxAe0qw0+wJ1Boc9PE3p+PjdU7Qo5t3J5sXDOOVeUd0NlUK32m5YyBrFoyaKaGYMoVa2M\nWZSi5/jGokW/7bd9rQ5C1+r/vcHgxcJVTDljEKXM9xTyXE9pG+hcUqWMb4CPcYQok8VC0qnAdUTd\nZDfkb7U6VIR+rnfH0o7u+2XnlTLf02DP9VSqLF9nUYm5pDo62rvv5x2a0P/vlSNzxULSMOC7wElA\nJ/ArSXea2XPVjWzwPPnkk5n5g71pwU28Yf1/488r7Fbq70jh+V+uYMWLXcHODLth7ZrMFotS5Y9E\nCo9Cnn9uCSs6F72rbeGMtqWclhu1f4aGDPRIZun/XtZkrlgARwMrzSwHIOlW4AxgyBSLV155pXij\nQdL1ShfvP+v9idoWdiv1d6Tw4g4bGT3m+KrODJumLa+/Xu0QKiZ/JFJ4FPLin57q9Yik8Cik1G6r\n9vaLyw+2ArL0fy9rslgsxgFrCtZfJCogrsqKjSuEeqTgkvF7a4Qti8Wiqn7xiyU8+uiTidvX1b2H\niy6aVtEYVq9eXdH95Q0bNowtr25hza+jWvyrpc/w2mtv9tneNm/jrU1v8vth0betXK6TDxxxbp/t\nkx4pbNrw8kDCrxlDNb9Sx0OW/fgnbHw9OnsuK8Ulrf97IVAa0zmUQ9KHgCvM7NR4fQZghYPckrIV\ntHPO1Qgz00Cel8VisQPQQTTA/QdgCfApM1tR1cCcc24Iy1w3lJm9LenvgYW8c+qsFwrnnKuizB1Z\nOOecy55Mzw0l6VRJz0n6raTLe3l8qqTfxD+PSjqsGnEOVIL8psS5LZO0RNIx1YhzoIrlV9DuKElb\nJJ01mPGVI8F7N0nSK5KeiH++Wo04ByrJeydpcvy3+YykBwc7xnIkeP++HOf2hKSnJW2VVFeNWAci\nQX6jJN0l6ck4v+aiOzWzTP4QFbLfAQ3ATsCTwME92nwI2D1ePhX4ZbXjrnB+IwuWDwNWVDvuSuZX\n0O5+4GfAWdWOu4Lv3STgrmrHmmJ+uwPPAuPi9T2rHXcl8+vR/uPA4mrHXeH3byYwO//eAeuBHfvb\nb5aPLLovzjOzLUD+4rxuZvZLM9sYr/6S6BqNWpEkv80Fq7sC2wYxvnIVzS92KfBj4I+DGVyZkuY2\noLNOMiBJflOBn5jZWgAzq6XzhZO+f3mfAn40KJFVRpL8DNgtXt4NWG9mW/vbaZaLRW8X5/VXDP4G\n+HmqEVVWovwknSlpBXA3cMEgxVYJRfOTVA+caWb/QW19sCb92/xwfJj/P5IOGZzQKiJJfn8BjJb0\noKRfSTpv0KIrX+LPFkkjiHotfjIIcVVKkvy+CxwiqRP4DfCFYjvN3NlQAyHpBOB84Nhqx1JpZnYH\ncIekY4GrgI9UOaRKug4o7E+tpYJRzK+B/cxss6TTgDuIPmBDsSNwBHAisAvwmKTHzOx31Q2r4j4B\nPGpmoc0D8lFgmZmdKOkAYJGk95tZnzdCz/KRxVpgv4L1feJt25H0fmAuMMXM/jRIsVVCovzyzOxR\nYH9Jo9MOrEKS5HckcKuk54G/BuZImjJI8ZWjaG5mtinfjWhmPwd2Cuy9exG4z8zeMLP1wMPA4YMU\nX7lK+b93LrXVBQXJ8jsf+G8AM1sFPA8c3O9eqz0Y088gzQ68M0izM9EgzYQebfYDVgIfqna8KeV3\nQMHyEcCaasddyfx6tL+R2hngTvLejS1YPhpYXe24K5zfwcCiuO1I4GngkGrHXqn84na7Ew38jqh2\nzCm8f3OAWfHyWKJuq9H97Tez3VDWx8V5kv4uetjmAl8DRgPXSxKwxcxqYtLBhPn9P0mfBd4CXgfO\nrl7EpUmY33ZPGfQgByhhbn8t6XPAFqL37pzqRVyaJPmZ2XOS7gOeAt4G5prZ8iqGnVgJf5tnEh09\n1dQ0wgnzuwqYJ+mp+GmXmdmG/vbrF+U555wrKstjFs455zLCi4VzzrmivFg455wryouFc865orxY\nOOecK8qLhXPOuaK8WLghR9LbBVNP3yZpeInPf63E9jf2Nv16PI353fHyJyRdFi+fIan/q2mdG2Re\nLNxQ9GczO8LMDiO6aO7ing3iizz7UsmLkwzAzO42s2vjbWcCh1bwNZwrmxcLN9Q9AhwoqSG+WUyb\npKeBfSR9StJT8c/VBc+RpG/GN/1ZJGlMvPFv4ptULZN0e48jlo/Es7M+J+n0nkFImibpO5I+DEwB\nro2PfvaX9OuCdgcWrjs3WLxYuKFIAJJ2BE4jmtcI4CDgu/ERx1bgamAyMBE4qmCSw12AJWb2f4gm\n0Lsi3v4TMzvazD4APAdcWPCaDWZ2FNGNdL4naede4jIzewy4C/in+Ojn98Ar8YSZEE0A94Oysndu\nALxYuKFohKQngCVADrgh3r7azH4VLx8FPGhmG8xsG3ALcHz82DZgQbz8QyB/u9v3S3o4nm9nKtt3\nJS0AsGgK71UUm+FzezcA50saRjTH1PwSnutcRWR2IkHnUrTZzI4o3BAPUfy5R7uk99fIj2HcSDRV\n/jOSphHdWrVnm/x+Sxn3+AkwC3gQWGq1NRW/C4QfWbihqK8iULh9CXC8pNGSdiC6tWZ7/Ngwovtv\nAHyaaNwDolvfrpO0U7y90CcVOQAYD3T0E99rwKj8ipm9CdwH/AdRQXJu0HmxcENRX9/qu7eb2Tpg\nBlGBWEb0jf5n8cObgKPjgfDJwP8Xb/8aUZF5BFjRY98vxI/9D/B3ZvZWP/HdCvyTpF9LGh9vu4Vo\nKvCFxZJzLg0+RblzNUDSl4BRZjar2rG4ocnHLJzLOEn/DexPdL9r56rCjyycc84V5WMWzjnnivJi\n4ZxzrigvFs4554ryYuGcc64oLxbOOeeK8mLhnHOuqP8fQllUv3enIC4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xf1e9b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure()\n",
    "test_data[\"p_up\"].hist(bins=20, alpha=0.4)\n",
    "test_data[\"p_down\"].hist(bins=20, alpha=0.4)\n",
    "plt.title(\"Distribution of Probabilities\")\n",
    "plt.legend([\"p_up\", \"p_down\"])\n",
    "plt.ylabel(\"Frequency\")\n",
    "plt.xlabel(\"Probablity\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRADING STATS\n",
      "AVG Monthly Return :: 0.40%\n",
      "STD Monthly        :: 3.09%\n",
      "SHARPE             :: 0.45\n",
      "MAX DRAWDOWN       :: 10.45%, 30.0 months\n",
      "Correlation to SPY :: 0.01\n",
      "NUMBER OF TRADES   :: 563\n",
      "TOTAL TRADING DAYS :: 4268\n",
      "SPY MONTHLY RETURN :: 0.76%\n",
      "SPY STD RETURN     :: 4.44%\n",
      "SPY SHARPE         :: 0.59\n",
      "SPY DRAWDOWN       :: 55.37%, 8.0 months\n",
      "0.104531465492\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEZCAYAAAB4hzlwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Wd0VFUXgOH3UKV36R2kSO9SQ5MqCCIooiAKCvaKDUFR\nFBBEkA8FaSIoYENUUFpQeu8QSuhVegup+/txJiFlEiZkJjNJ9rPWLDLntj0Tcvc95Z5rRASllFIK\nIJ23A1BKKeU7NCkopZSKoklBKaVUFE0KSimlomhSUEopFUWTglJKqSiaFJRSSkXRpJDGGGOuGmOu\nOF7hxpgb0coeNcYMMcaEON5fMMasNMY0cLKfUo7tJzhZFhFtn/8ZYxYbY7rHWme5Maav4+dmjm2+\njLXOv8aYJ6K9L2SMmWSMOeHY9wFjzFRjzD0JfN53jDGBjvWPGmO+j7bM3xgT5Fh21hjzo+MYTxlj\ndhtjMkZbN58x5owx5v54jlPUGPOdMeac47OvNcZ0iLa8pOMzpou13TRjzDBjzNvRvrMgY0yY4+er\nxpgd8RwzkzHmE2PMEWPMdWNMgDHm9VjrRP+MkfuvH8/+njLG7DHGXDbGnDLG/G6MyRZrnaGOz1E3\nVnlvR/noWOWdHeVTY30Pkf8HA40xg6KtH2GMKeMktt7RvpPon6WQs8+i7pwmhTRGRHKISE4RyQkc\nATpEK4s8Yf7gWJ4f8AfmOdnVE8AFoEf0k2fkYYBqjn1UAGYAXxpjBicQ2nXgcWNMCWcLjTF5gdVA\nFqCRY9+1gBVA63i26Q08BrRwrF8HWBorzoGOZfcAeYAxIjIFOA4MibbuWOB3EfnbyXHyACuBm0Al\n7Pc2FphtjOka63jOiIh8Evl7AJ4FVjt+JzlEpGo82/0INAfaAjmAx4H+xpgvnH3GaL/ndU4+QzPg\nY6CHiORyfI45To75OHAe+/uP7SDQPVbiewIIiP15gVyOz9oTeD9ask3obtrI7yT6ZzmdwPrqDmhS\nSNuM4+WUiEQAs4Aixph8sRY/AbwHhAIPxLdfEbkgIt8BA4B3HCdQZy4B04Gh8Sx/FbgsIo+LyGHH\nvq+IyAwRiVNbcagD/BVt/bMi8o2TWBGRS8BPQBVHeT9ggDGmmjGmDfbk+2oCsV0VkadF5D8RCRaR\nH7An2THxbJMkxpiWQCugq4jsEZEIEVkP9AKei3W1He/vOJo62JPudrDfh4jMFJHr0Y7ZFCgEvAg8\naozJEGsfp4EdQBvH+nmAhsBvzj6C4zhrgV3c+t5diVV5kCYFFS9jTCagN/bK8GK08iZAUeAHbC2i\ntwu7mw9kAOrFs1ywJ9GHjDHlnSxvCfzicvDWWuAJY8zrxpjasZtuojPG5AceAjYDiMgR4H1gGvA/\nYICIXI5n81bYhBLbXKBEPJ8nqVoB60TkZPRCR2I4jv2+EmMd0MbRPNTQ8buP7QlgAbdqjrEvBgT4\nllv/Hx4BfgVCnOzLABhjGgGVcXzvyvs0KShnehhjLgA3gKeAbo5aQ6QngD8dJ8nZQFvHSTVeIhIG\nnAPyJrDOWeAr4EMni/Njr0QBMMY8YIy56GhXXhTP/mYBLwD3Y5vBzhhj3oy12njHZ90CnABei7b9\nBGxNaKuILEjg4+UHTjkpPxVtubvFd8zI40Y/5jhj+4cuGmM2OttARFYCXYGawO/AOWPMaGNM5Mk7\nC/AwMMvxu/wR501IvwLNjDE5Hcu/dbKOAf4zxpwHJgGDRMQ/wU9r3ef4HJGfZb8L26hE0qSgnJkj\nInmBu4Gd2KYFAIwxd2FPDrMhqvp/DNs2HC9HU0MBbK0jISOwV6zVYpWfBwpHvhGRBSKSB3gFcHZV\nG7ne9yJyP5Ab21Y/zBgTvQ/iBRHJKyLFReQJEYkd3x5s80ZCzkWPLZrC0ZaHOX6O3f+SEZt4Eiu+\nY0Ye91y09y86PmMeEakTzzaIyF8i0tnxu+8M9AGedizu6ohzoeP9bKB97GZFEbkJ/IFtWswrImuc\nHQrIJyL5ROTeBJr/Ylvj+ByRn8UTNbA0T5OCipeIXACeAYYaYwo6irsCOYH/OUaonAKKcPsmpAex\nJ5UNLhxzLDCMmJ2OSx37uCMiEi4iPwHbudV+7S5LsN9LbD2AoyKyH3v1HgqUirVOaWyH/50cs74x\npmj0QsfIomLE7FBPNBFZDizj1nfVG8gOHHX8zudimwOdXQzMxPazzEzgENp34KM0KagEicg+YBEQ\nOWywNzAFqApUd7waAzWMMffG3t4Yk8cY8xjwJfCpiFyMvY4Tn2M7KCtFKxsD5DHGzIzsRDXG5ABq\nxLcTxzDG9saY7MZqh22/XutCDInxOZDLGDPFGFPQGJPZGPMo8DbwOkR12v8EfGyMyWuMyeBYpxK3\nrr5dJiJLsSf+n4wxlY0x6YwdOjwT+J+IBCZmf8aYTsaYHsaY3I739YBmwBpjTBGgBdAB+31XB6oB\nI3FyMSAikSPCvoy9LPJwtwkns+M7jHxFnqc0kSQDTQppm6sP0/gM6OcYLtoc+NwxkifytRl7Yot+\ngthmjLkC7Af6Ai+JyAeuHFtErmJPOHmjlZ0HGmCHfa507Hsz9up1QDy7ugK8g70Svwh8CjwbrUnD\nlc9/23UctZvG2OGyu7FNNy8DvUTkx2irDsQO490OnHG8by8i/7kQhzMPAcuxSfsqtv1+soi8mJj4\nHS5iR1ztM8ZcduxrhGMU1ePAZhFZGv33DowDqhpjKsfemYgsd4zociahmATbZHkDCHL828exrIGT\n+xRqu/j5lItMcjxkxxgzBegInBGRao6yPNhx0CWBw0D3yNEdxpi3sSeSMOzJJM7YcKWUUu6XXDWF\naTjGLkfzFrBERCpg2y7fBnBcdXTHVqvbYduutdqolFLJIFmSgmO4W+y25M7YO11x/BvZidgJe0dt\nmOOmo/3EP7ZdKaWUG3mzT+FuETkD4LhV/W5HeVHsEMdIJxxlSimlPMyXOpo937mhlFIqQbHnLklO\nZ4wxBUXkjGOmw7OO8hNA8WjrFXOUxWGM0USilFJ3QESc9tUmZ00h9uRrv3FrqFlv7Nw4keWPGDst\ncGmgHLA+vp2KyG1fQ4YMccs6rq7XrFkzjcuH43I1dlficuf3oHFpXJ48XvT1EpIsNQVjzGzAD8hn\njDmKnZL4U2CesXPqH8GOOEJEdhtj5mLHe4dip/1NUo3Az8/PLeu4ul6pUqXcti+NK3H7cmfsrsTl\nzu9B40rcehpX4o7n6nouZRhffdnwfc+QIUO8HYJTGlfiaFyJo3Eljjfjcpw7nZ5XfamjOdVwOSMn\nM40rcTSuxNG4EsdX40qWO5o9xRgjKTl+pZTyBmMMEk9HszdHH3lMqVKlOHLkTiaeTF1KlizJ4cOH\nvR2GUioFSZU1BUcW9EJEvkW/B6WUMwnVFLRPQSmlVBRNCkoppaJoUlBKKRVFk4JSSqkomhS8oFSp\nUhQsWJCgoKCosilTptCiRQsA0qVLR/Xq1WNsM3jwYPr27ZuscSql0h5NCl5gjCEiIoKxY8fGu87J\nkyf54YcfkjEqpZTSpOA1b7zxBqNHj+bKlStOl7/55pu8//77REREJHNkSqm0TJOCl9SpUwc/Pz9G\njRrldHnXrl3JlSsX06dPT97AlFJpWppNCsa455UUH3zwAV9++SXnz593Ep/hww8/ZNiwYYSFhSXt\nQEop5aI0mxRE3PNKinvvvZeOHTvyySefOF3erl07ihUrxldffZW0AymllIvSbFLwFUOHDmXy5Mmc\nOOH04XJ89NFHDB8+nBs3biRzZEqptEiTgpeVLVuWHj16MG7cOKfLmzVrRpUqVZgxY0YyR6aUSos0\nKXiBidUZ8f7773Pjxo2o8tjLP/roIy5evBinXCml3E1nSU3F9HtQSjmjs6QqpZRyideTgjHmFWPM\nTmPMdmPMLGNMJmNMHmPM38aYAGPMX8aYXN6OUyml0gKvJgVjTBHgBaCWiFTDPgnuUeAtYImIVACW\nAW97L0qllEo7vF5TANID2YwxGYAswAmgMxA53GYG8KCXYlNKqVTl74N/J7jcq0lBRE4Co4Gj2GRw\nWUSWAAVF5IxjndPA3d6LUimlUofwiHDeWPxGgut4u/koN7ZWUBIogq0xPAbEHjKjQ2iUUiqJZu+Y\nTdaMWRNcJ0MyxRKfVkCgiFwAMMb8AjQEzhhjCorIGWNMIeBsfDsYOnRo1M9+fn74+fl5NGCllPIl\nIsLNsJtcDr7MleArXA+5zo3QGwSHB1O/aH2yZcqGv78/S5Yt4cv1X9K1UlfWsjbe/Xn1PgVjTD1g\nClAXCAamARuAEsAFERlhjBkE5BGRt5xsr/cpJEC/B6VSt0MXD1H/m/pcDr5Mrsy5yHVXLrJlzEbW\njFkJiwjjWsg1fur+E5UKVGL06tH8c/Qf5j8yP8H7FLxaUxCR9caYH4EtQKjj30lADmCuMaYvcATo\n7r0olVLK94gIA/8cyMsNXuadJu84XWfqlqk0nd6Uj1t8zIhVI/Dv43/b/Xp99JGIfCAilUSkmoj0\nFpFQEbkgIq1EpIKI3C8il7wdpzutXLmSRo0akTt3bvLnz0+TJk3YtGkTM2bMIEOGDOTMmZPcuXNT\nq1Yt/vjjD86ePUuBAgX4559/Yuynb9++9OzZ00ufQinlTXN3zeX4leO80TD+juO+Nfuy5PEljFo9\nis4VOlO5QOXb71hEUuzLhh9XfOW+4MqVK5I7d26ZM2eOREREyM2bN2Xx4sWyY8cOmT59ujRp0iRq\n3fHjx0vWrFnl0qVLMnPmTClfvrzcvHlTRESWLFkiBQsWlPPnz8d7LF/+HpRSd+7CjQtS+LPCsuro\nKpfWvxl6U0LCQqLeO84NTs+rXq8ppDX79u3DGEP37t0xxpA5c2ZatWpFlSpV4qzbt29fgoKCOHjw\nIL169aJixYq8//773Lx5k2effZbx48eTN29eL3wKpZQ3vbXkLTpX6EzD4g1dWj9zhsxkTJ/RpXW9\nPfoozbnnnntInz49ffr04ZFHHqFBgwbkzp07znphYWFMnjyZHDlyUL58eQAmTpxI9erVCQgIoGrV\nqjz88MPJHb5SygtuhN5gQcAC1p1Yx9rjazl25Rg7BuzwyLHSbFIwH7hnGmoZkrjRPTly5GDlypWM\nGDGC/v37c+rUKTp06MCkSZMAWLNmDXnz5iVDhgyUK1eOX3/9lRw5cgBQtGhRPvzwQwYNGsTBgwfd\nEr9SyrftPbeXbnO7UThHYVqWbsnHLT6mbtG6ZM+U3SPH06mzvWzfvn089thjlC9fnjZt2jBlypQ4\nHcrRrVixgscff5yjR4/edt8p6XtQSsU1Z+ccnl/4PJ+0/ISnaj7ltmeq+OyQVGWbk/r06cOkSZNo\n06aNt8NRSrmJiBAhEaRPl97lbc5eP8vfB/9m86nNbDy5kRNXT/BXr7+oVbiWByONSTuak1lAQABj\nxoyJeibzsWPH+P7772nQoIGXI1NKhYSHcOrqqTjlQaFBvL/8fa6HXE9w+8OXDvPKoldoMaMF+Ubm\no8TYEiw+uNjl43ef153ZO2Zzd7a7ebfJu2x7dluyJgTQpJDscuTIwbp166hfvz45cuSgYcOGVKtW\njdGjR3s7NKVStbCIsASbU//c/ydV/leFe768h3m75kWVB4UG0fmHzny2+jN+3P1jgscYuWokp6+f\nZlCjQex+bjfTO0/nyflP8tpfrxEcFpzgtsevHGf7me380uMX3mr8Fm3KtfFYv0FCtE8hFdPvQalb\nmkxrQraM2Zj8wGSK5yoeVb7jzA7eXvo2+87vY2zbsRTJUYSOszvy2n2vMaDuAB784UHyZMlDl4pd\nmLhxIst7L4/3GOXGlePnHj9TrWC1qLLzN87Tb0E/Tl07xaq+q0hnnF+Lf77mc3ac3cHUzlPd96Hj\noY/jVEqlaVtObeHo5aM0LtGYWpNqMXnTZBYdWETrma1p810bmpdqzs6BO2lfvj01CtVg9VOr+WbL\nN1SeUJlcd+ViZpeZPFjxQXae3UngxUCnxzh44SDXQ69T9e6qMcrzZc3HT91/IiwijAUBC+KN8Ydd\nP/BIlUfc+rnvhNYUUjH9HpSy+i/oT8lcJXm36bvsPLuTfgv6cTPsJq80eIVHqjxCpvSZ4mxzMegi\nM7fPZGDdgWRIZ8fkvLjwRfLclYcPmn8QZ/2vNn7FmuNrmPHgjDjLAObtmsfYdWNZ1XdVnGWBFwO5\nb8p9nHj1RNSxPCmhmoImhVRMvweVFhy6eIgP//mQrBmykidLHkrnLk3fmn2jhm9evnmZUl+UYs9z\neyiUvVCSjrXl1Ba6zOlC4EuBcZqBuszpQrdK3Xis2mNOtw2PCKfClxWY/uB0GpdoHGPZJ/9+wvEr\nx5nQYUKS4nOVNh8ppVKlK8FX6Ph9RwpkLUClApXInD4z49aPY9TqUVHrfLvtW9qUbZPkhABQs3BN\nct+VG//D/jHKQ8NDWX5oOa3Lto532/Tp0vN6w9cZuWpknGXf7/zeJ5qOQO9TUEqlUOER4Tz606M0\nLdGUEa1GRNUM+tToQ71v6lGnSB2al2rOxI0TmdhhotuO26dGH6ZvnU6L0i2iytafWE+ZPGW4O1vC\nTw7uXb03Q/2Hsvu/3VEzlu46u4sLQRdoVKKR22JMCq0pKKVSpEFLBnEz7Cbj2o2Lcadv8VzF+a7L\ndzz282PM2jELYwxNSzZ123Efq/oYvwX8xpXgK1Flfx/8m/vL3n/bbbNkzMLz9Z5n1OpRhEeEczPs\nJrN2zKLHvT3iHZWU3FJlTaFkyZJuux08JStZsqS3Q1DKI77e+DXzA+az7ul1Tmf/bFmmJS/Xf5nH\nf3mccW3HufV8UCBbAbrf251eP/fip+4/kTF9Rv46+BfDWw53afuBdQdSaUIlMg7LSKb0mciROQdL\nn1jqtviSKlV2NCulUofAi4HkypyLfFnzRZX9uPtHXlz4Iv8++S9l85aNd1sRYdy6cfSt2ZccmXO4\nNa6Q8BC6zulKjsw5GN9uPGW+KMN/b/xH5gyZXdo+QiIwGK9dvKa50UdKqZTvRugNKk2oRFBoEB82\n/5B+tfqx4sgKHvnxEf5+/G9qFKrh1fiCQoPo+H1Hzt04R7Gcxfij5x9ejScxdPSRUirFGbVqFPWL\n1mfx44uZtWMWdSfX5ZEfH2Hew/O8nhDA9g/Mf2Q+ue/KTZeKXbwdjtt4vaZgjMkFfANUASKAvsA+\nYA5QEjgMdBeRy0621ZqCUqnQ0ctHqfl1TTb330zJ3CUREb7f+T35s+Z3qUM3OUWeg1JSP6ZPNx8Z\nY6YDK0RkmjEmA5ANeAc4LyIjjTGDgDwi8paTbTUpKJXCnb1+lo/++YieVXvSoJidLbjHjz2olL8S\nQ/2Geje4VMpnk4IxJiewRUTKxirfCzQTkTPGmEKAv4hUdLK9JgWlUrDAi4G0+a4NDYs3ZGngUpqX\nbk6H8h14c/Gb7H1+L1kzZvV2iKmSL/cplAbOGWOmGWM2G2MmGWOyAgVF5AyAiJwGEr4jRCmV4mw7\nvY0m05rwSoNXmPHgDPY+v5eSuUrS59c+fHb/Z5oQvMTbNYXawFrgPhHZaIz5HLgKPC8ieaOtd15E\n8jnZXoYMGRL13s/PDz8/P88HrpRKkrXH19L5h85MaD+BbpW7xVh2PeQ62TJl81JkqZO/vz/+/v5R\n7z/44AOfbT4qCKwRkTKO942Bt4CygF+05qPlIlLJyfbafKRUCrPx5Ebaz2rP9Aen0758e2+Hkyb5\nbPORo4nomDHmHkdRS2AX8BvQx1HWG5if/NEppdxt6+mtdJjdgckPTNaE4KN8YfRRdeyQ1IxAIPAk\nkB6YCxQHjmCHpF5ysq3WFJRKAS4GXeSXvb/w7rJ3+bLdlzxU+SFvh5Sm+ezoo6TSpKCU7woKDeLn\nPT8za8csVh1bResyrRlQZwAty7T0dmhpniYFpVSy2X9+P2PWjGHu7rnUK1qPx6s9zgP3POD2+YfU\nnUsoKaTKWVKVUt4RIRF0nduV9uXas/WZrRTPVdzbIalE8vZ9CkqpFOjQxUPU+roWu//bHaP85z0/\nkyVDFj5t9akmhBRKk4JSKlEuBF2g3ax2FMhWgH4L+hEhEYCtJXyw4gOG+g1NUfMAqZg0KSilXBYc\nFkyXOV3oUL4DCx9bCMD/NvwPgJ92/0TWjFlpV66dN0NUSaQdzUopl4gIj/38GCHhIcx9eC7pTDr2\n/LeHJtOasLH/RjrO7sio1qNoV16Tgq/TjmalVJJ9tvozDlw4wIo+K6KeJ1ypQCVeqv8SzaY3o3D2\nwrQt19bLUaqk0uYjpdRt/XPkH0avGc2P3X8kS8YsMZYNajyIkrlK8nGLj7UvIRXQ5iOlVIJOXztN\n7Um1mdJpitYEkoEIrF4N48ZB1arw3nvuP4bPzn2klPJtYRFh9PypJ0/VfEoTgoeFhMCsWVCvHvTu\nDfXrwzffwI8/Jm8c2qeglIrXtC3TCIsIY0izIbdfWd2RM2fgq6/s6957YfBg6NgR0qWDZs2gbVuo\nVMkuSw6aFJRSToWGhzJ85XBmdZ1F+nTpvR1OqhQeDg0bQsuWsGRJ3BN/7drw2WfQpQusXw+5c3s+\nJm0+Uko59d327yibpywNizf0diip1uLFkDcvTJoUf02gd29o0wYaN4bRo+HYMc/GpDUFpVQcYRFh\nfPzvx0ztPNXboaRqkyZB//63X++LL2xNYs4cqFED7rkH2re3TUu1a9umJnfR0UdKqThmbpvJlC1T\n8O/jn6jtQkNtM0eFCpA/v2diSy1OnYLKleHoUciRiAlkQ0LA3x/++gsWLoQLF+DZZ+G556BAAdf2\noVNnK6VcFh4RTuX/VWZih4m0KN3CpW1+/hlmz7ZXs0WL2hPXkiVQsmS0/YbDxo2wdSts2waBgfZk\nmDu33eatt+Cuuzz0oXzQ8OFw5Ah8/XXS9rNnD4wdC3PnQo8eMGrU7ZOMDklVSsUxb9c8Ai8Gximf\nsW0GBbIWoHmp5rfdR2goPP88vP02PPAABATArl3wwgvQpIl9D/bKtnZtePJJWLfO1iReeAEefhjq\n1oUNG+yyiAg3f0gfFREBkye71nR0O5Uq2cQSEAA3bthO6eDgO9+f1hSUSoMuBF2gxOclKJu3LGuf\nWht1l/LBCwdpMKUBSx5fQvVC1RPcx3//QffukDWrHV8fe2TM1Kn2xqu6dWH7dhg5Erp1A2c3PQcF\nQatWdgjm8OGJ+yyhoXDwIFSsmLjtvOmvv+Cdd2DTJvfuNzzc/k4yZrQ1t/j6GrSmoJSKYdKmSTxU\n+SEq5a/ES4teAuwQ1J4/9+S9Ju8lmBCuXLHNFTVrwn33wW+/OR8q2bevvYJt1sw2cTz8sPOEAJAl\nC8yfD/Pm2SvonTvhk0+gaVPo2hWmTLFt8LEFBtpROTVrwvLld/JNeIerHcyJlT69TdCnTsFLL9m7\no2Pbvz/hffhETcEYkw7YCBwXkU7GmDzAHKAkcBjoLiKXnWynNQWlEikkPITSX5Tmz55/UjpPaepM\nqsP7zd5nz3972HJ6C3/0/CNqDqODB+Hjj+1JO3dumxBmz4bWreHVV+3dt+60fz80amSP98AD0KED\nnDsHv/8Of/8NZcvaG7s6doR9++yJ77337HQQjz5qE0Plyu6Nyd327LHJ7uBByJnTM8e4dAmaN7d9\nNR9/DNWr21rE2LE22Z4/7+MdzcaYV4DaQE5HUhgBnBeRkcaYQUAeEXnLyXaaFNzs4kXIk8fbUdgr\nwGnToHBhKFPGNg2UKuXtqFKHmdtmMmPbDJY8sQSAbae30eLbFmROn5ktz2yhYPaCgG2WadTIvsqU\ngcuX7ZX+449DiRKeiy80FDJkiFurCA2FVatsgvj9d9s08t13UKuWXf7ttzBkCKxZA4UKeS6+pOrY\nEVq0sEnVk4KDbU1t+HDw84NDh2xT3zffQLly8ScFRMSrL6AYsBjwA35zlO0FCjp+LgTsjWdbUe5z\n9KhIliwi69Z59jjffSfy11/Ol0VEiHz9tUi+fCIvvyzSv79I69b2/Wef2eWRrl4Vef55kTlzPBtv\nSnfwwkGJcHxxERERUn1idflz358x1vlt72+y8sjKGGWDB4u0bRvzO/d1Q4eK1KsnEh6e9H2Fhoos\nXmz/r/r7i2zbdvttLlwQ2b5dJCzM+fK//xYpW1bk5s2kx+eqq1dFRowQ+eabW9+L49zp/Jwc34Lk\negHzgBpAs2hJ4WKsdS7Es617v7007sknRYoVE3nmGc8dY8QIkdKlRUqWFHniCZFz52x5aKjIqlUi\n7dqJ1KolsmtXzO2OHBGpWlVkwAC77pYtIvfcI/LYY3Zf77/vnhNBajNm9RhJ90E6aTmjpWw8sVGW\nBi6VSl9WkvCIhL+s1atFChYUOXkymQJ1k4gIkZo1RRYtStp+1q2z+6lZU6RVK5EmTez3MXFi/Nv8\n84/9+ylTRiR3bpH27e36oaF2eViYSJUqIj//nLTY3MFnkwLQAfjS8bNfAknhfDzby5AhQ6Jey5cv\nd/uXl1bs2CFy9932ZJwnj8j16+7df0SEyJAhIhUqiBw/bq9eXnxRpFAhkQceEMmVS6R6dZFPPhEJ\nCXG+j8uXRe6/314J5s8vMnu2LT99WuS++0S6dXN/3CnZpI2TpOTnJeXghYMyccNEKfxZYbl71N0y\nedPkBLe7etVezf70UzIF6mZffSXSteudbXvunMjAgfb/5Xffxawl7dtn/0b++SfmNuHhIsOH26Tx\np6MCduaM/f5atrT/r9essTXgZs28U/Navnx5jHOlLyeF4cBRIBA4BVwDZgJ7YjUf7Ylne898gx7y\nxx8iJ05459jXr9s/9vh07Cjy+ef257Zt7R+EOw0ebK/0T5+OWb5+vT25xy6PT0iIyMiRIvv3xywP\nChLp0kVk0CD3xJvSzdo+S4qMLiL7z9/6oq4GX5Upm6dIUGiQ021u3rRXtiVK2Ga5lOryZXulnpha\nzuXLIh98YJspBw60zUDOLFpkE8aRI/bkvmyZSKNG9nXsWNz1IyLs/+/ChUWyZxfZtOnOPpO7+WxS\niBFIzOajkcAgx8+DgE/j2cb935aHHD4ski2bvQpv08aedI8e9fxVw5Yt9j95njy2meX48bjrrFgh\nUqrUrXZLSsg4AAAgAElEQVTOOXNEWrSIu97Vq7bq26ePyPjxrscQWQv57787+ggu27RJpFy5lNUG\n7gnfbv1WCo4qKDvO7Ih3nZMnbRJt0MDW1Hr3tk0f7drZq9qUrl8/kY8/dr7s6FGRjz4S6dVL5OGH\nRTp1sv8/e/USOXDg9vseNUqkWjWRpk1FypcX+fbbW01E8bl4MelNWu6UEpNCXmAJEAD8DeSOZxsP\nfF2eMXq0yNNP2yv22bPtlXmhQrbZpFEjkfnz3XMcf3+RV1+1f9ylSokUL247344eFfn0U9umGf0q\n6Pp12xwTvWZw86a9Yjp0yL4PC7NNPTly2PbVTz8VyZtX5Px512Jq107kiy/c8/kSEhFhP2/s/oi0\nIiIiQoatGCYlPy8pu87G/yUsXGj/7w0ZYvtxfv1VZPJkkQ0bki9WT9uwwf7/j97P9Ndf9oIsb16R\nZ58VmTFD5Icf7IXOnj2u7zuyKdSVZOCrUkRSuJNXSkoK993n/Erh7FmRBQtEihSxiSMpV7l79ti2\n9uHD7R96QEDMURARESKvvGKT0MWL9mq/cGF7hRS7k/b55+1//Js3RR56yLaNRk8CTz1lO3dvZ/Fi\ne/UeHHznnysxXngh/ivE1GbVKvt5Bw4U6fdsiNQd9rRUm1BTTl5x3m4SFmab14oXtxcPqV2tWvZv\nLiLC1gyKF7cXPzdueDsy79Ok4GVHj9or7/g6UEVsG2WVKrdG1yRWcLD9I0hodISIPfn36iVy1112\ndMTmzc7X27zZNje1bGmTQuwhdAcO2M906VLCx6pRQ2TevER9lCRZulSkbt3kO563XL9u2/6HDLHJ\nve3Id6XQmy0kb+Er8sEHNulHFxRkm0r8/G6N+ErtvvrK1sh79RKpXdt7/Xm+SJOCl40da4d73k7k\n6Jqnn078MQYNsm3DrtQ0QkJsO//t1Klj7xOIb8z1E0+IDBsW//YzZtgaUnK28YeE2OYBZ51+iREU\nFDfhnT5tE9y4cbavxptDYIcOFene3f4ccC5A8o3IJ8cvH5d9+2yfT548tslv/377Ofz87Ois5Bwf\n721XrtjOXR2VFleSkwLQEOgJPBH5cmU7T79SSlJo3NiOPHLF1au2uSUxwwGXLbPNT2fP3ll88Umo\nZiMisnevba66ciXudtOn2yF6K1c639aTHn9cZMKEO9s2IkJk1iz7fWbLdmuobPnydkRLhw42aZcv\nbz97t2622W/1aptIEiMk5M6S15EjtpZmR8BEyP0z75fPVn0WY51jx0TeftvGWLCgyHPPxZ/cU7Mj\nR/T+FWcSSgq3nebCGDMTKAtsBcIdxSIiLya4YTJICdNcnDwJVarYCaoyZ3Ztm3XroFMn2LzZzl2S\nkHPn7G3+kybZpzAlt5497fwtrVrZybiOHoXPP7fTIrz3nr2dP7n9/LN9CPrff8csF7ETq73+up1b\np2BB+ypb1k7lXKwYjB9v5/eZMAEaNLAPMDlyxM46ee+9MWedPHYMVqyw0yqsXWvnspkxAzp3vn2M\nIna6iHnz7Bw/b7wB9eu79vl69LDTJQ8dCj/t/okh/kPY8swWMqbPGGfdGzfsswsaNIh/MjqV9iTp\nITvGmD1AZV88+6aEpDBhgj3Jf/tt4rb78EP49187xW5809+K2BNKpUr2wRreEBhoT/4hIXbCrRw5\nYMAAO3umt1y/budMOnr01uydN27YJ1OtW2cfaZg3L5w9a5P1gQN2crWDB+33+cwzNsEl1qZNdl6b\n4cPtswES8uGH8Mcfdg6f77+HMWOgeHH73XXtGv/DZlasgCeesJOqRWS4RuUJlZnZZSbNSjVLfMAq\nzUpqUpgHvCgiTiau9a6UkBT8/OzEV506JW67sDA75XDTpnaSL2cniTFj7NOW/vkHMmVyS7ipRqdO\n9mHnlSvDjh22hlClip0gLHt2zx03IMAe9/nn4eWX7cRusc2ebefSX7v21sRtYWF26uhJk2wNsUcP\nWwMsW9bWFv/91/6uV6+2FxhVmwYy8I+BFMhWgJldZnruA6lUKalJYTl2bqL1QNTzfEQkkac59/Nm\nUggIgFy5Ep6N8cAB+4CRU6fu7DGDR4/aq9vNm+G11+wVbLZsdtn69faqdP16nT3Ume+/h4EDbZNP\n1ao2OXfvnjxNKMeP2yakbdtszSlfPttMVbSofYbuvHmwdKmNy5lDh2xtZu9e+3/oyBE7RXX37uB3\n/3W+3PIpEzdO5LX7XuPV+14lcwYX2yWVckhqUnBaLxWRFW6ILUm8mRR69LAn4xEjnC8PD7dX+t26\n2SvGpNi61TZJ/PabPcEUKGD7Kr76yjY1KN8UEWHntT93Ds6csb+zEyfsVNSu9h/E1va7tmTPlJ2x\nbcdSLGcx9was0ow7TgrGmPTAEhG5/cNavcCbSaFGDdvuHN/j9EaOhIUL7RVhfH0CiXXzpj3BnD1r\n30fOI6/Shq2nt9JhdgcOvXSITOm1vVDduYSSgpMWz1tEJNwYE2GMySVOnnyWVkVE2CdEGWNP0vnz\nx1y+fbvt+N240X0JAWwTVLFi9qXSnjFrxvBCvRc0ISiPSjApOFwDdhhjFgPXIwt9YUiqtxw/bodh\n1qkDy5bZtt5IISF2dMjIkVCypPdiVKnLiSsn+H3f73zR9gtvh6JSOVeSws+Ol3IICLDj2lu3hsWL\nYyaFadNsp2KfPl4LT6VC49ePp1e1XuTJ4gPPSlWp2m2TgojMSI5AUpLIpNCqlR0WKmKbkkTgf/+D\n0aP1RiHlPtdCrvHN5m9Y32+9t0NRacBtk4Ix5hAQpzdXRMp4JKIUIDIpVKpkHyZ+8CCUK2cfKh4c\nDC1bejtClRIEhQZx8upJyuYtm+B6U7dMxa+UH2XypNk/OZWMXGk+qhPt57uAh7HPO0izAgLslBLG\n2NrC4sU2KUyYYO9I1VqCup2wiDC6zevGskPLGN9uPE/XejrOOnvP7WXihol8u/1b/ur1lxeiVGnR\nbcfGiMj5aK8TIjIW+2zlNCuypgC3+hVOn4ZFi6B3b+/GpnyfiDDg9wFESAQb+m1gzJox9F/Qn+Cw\nYA5eOMj4deNpPqM5ftP9yJE5B9ue3Ua9ovW8HbZKI1y5eS36aPh02JrDABGp7snAXOGN+xRu3LDz\n5ly7ZqcwOH3aNiO9/LK9MWnSpGQNR6VAH674kPkB81nRZwXZM2XnavBVnpz/JEsCl3BXhrtoX749\nHe/pSIfyHfRuZeUR7pjmIlIYcAgYLSIB7gvxzngjKWzbBo8+Crt33yqrWtX2K6xebW9qUyo+UzZP\nYfjK4azqu4pC2W/NkSIiBF4MpHSe0qQzbry5RSkn7vjmNYenRCQw1g5LuyWyFCggACpWjFnWurW9\nb0ETQtogIpg76DhaELCA95a/x4o+K2IkBLB/pLfrcFYqObhySfKji2WJZowpZoxZZozZZYzZYYx5\n0VGexxjztzEmwBjzlzEmlzuO5w7R+xMivfMOfPedd+JRyWvG1hnkHpGbNt+1YeSqkez5b49L260+\ntpq+v/Vl/iPzuSffPR6OUqk7F29NwRhTEbgXyGWMiT7tWk7sKCR3CANeFZGtxpjswCZjzN/Ak9g5\nl0YaYwYBbwNvuemYSRIQYEccRZc/f9ypLlTqc/DCQV5f/Dp/9vyT/278x7JDy2g4tSFHXj5Czsw5\n491u19lddJnThZldZmqHsfJ5CTUfVQA6ArmBB6KVXwX6uePgInIaOO34+ZrjgT7FgM5A5OysMwB/\nfCgpPPect6NQyS0sIoxev/TivSbv0ahEIwAerPggx68cZ96ueTxV66k421wJvsKnKz/l601fM67t\nONqW88Kj8ZRKJFc6mu8TkTUeD8SYUtiTfxXgmIjkibbsgojEuTciuTuaRewzFA4ftiOQVNrx4YoP\nWXl0JYt6LYrRETx/73zGrB3Dij4xZ5KftX0Wr/39Gu3Kt2NY82E6zbXyKUntaD5vjFkKFBSRKsaY\nakAnEfnIjQFmx/ZTvOSoMcQ+08d75h86dGjUz35+fvj5+bkrLABWrrRz32fMaIefZs6sCSGt2XBi\nAxM2TGDLM1vijAxqV74d/Rb049DFQ5TOY8dfHLp4iBcXvciSx5dQs3BNb4SsVAz+/v74+/u7tK4r\nNYUVwBvA1yJS01G2U0SqJDHOyP1nAH4HForIF46yPYCfiJwxxhQClotIJSfberSmEB5un3TWpw9M\nnGifj/veezZRqLRBRGg6vSl9a/TlyZrOH7z84sIXyZclH0P8hgDQ86eeVMxfkfebvZ+coSrlsoRq\nCq6MPsoqIrFn4gpLelhRpgK7IxOCw29AH8fPvYH5bjyeyw4fhjx57JxG48Y5H3mkUreFBxZyIegC\nT1R/It51elfvzbfbv0VE2HhyI/6H/Xn1vleTMUql3MeV5qNzxpiyOJpwjDHdgFPuOLgxphHwGPZ5\nDVscx3gHGAHMNcb0BY4A3ePfi+fs3QvVqtmHvTdsaB9u062bNyJR3hAhEbyz9B0+av4R6dOlj3e9\nWoVrcVeGu1h5dCVD/IcwpNkQsmfKnoyRKuU+riSF54BJQEVjzAnsHc293HFwEVkFxPfX1iqe8mSz\nZ4+dwqJUKfjxR2jeHN5919tRqeQyZ+ccMmfIzIMVH0xwPWMMvav3ZuCfAwkND3U6EkmplMKV5ykE\nAq2MMdmAdCJy1fNh+Ya9e+3T1cDWFDZvhnv0vqM0ITQ8lMHLBzPpgUku3b3cq1ov3l76Nj93/5kM\n6Vy51lLKNyXYp2CMSW+MyQ8gIteBYGNMP0dHcKq3d2/MKS3uvdeOQlLJ79TVUyTn8OOpW6ZSOk9p\nWpRu4dL6RXIUYceAHXSq0MnDkSnlWfEmBWPMI8AFYLsxZoUx5n4gEGiP7QdI9WInBeUdf+7/k2Kf\nF6PxtMb8se8PjyeH4LBgPv73Yz5qnrhR15ULVL6jOZGU8iXxDkk1xuwEHhSRA47ps9cA3URkQXIG\nmBBPDkk9dw7Kl4cLF/ShOd508MJBGk5tyI8P/8jJqycZvnI4BsPEDhO5r/h9HjnmVxu/Yn7AfBY+\nttAj+1fK2+705rUQETkAICKbjTH7fSkheNqePbaWoAnBe26E3uChuQ8xuOlgmpRsAkD3e7szb/c8\nHpzzIM/UfobBTQeTMb372vRCwkP4ZOUnzOk2x237VColSSgp3G2MiT7YOnf09yIyxnNheZ82HXlH\neEQ4V0OuciX4Cu8sfYcqd1fhubq3JpsyxtD93u40KdGEJ+c/SaOpjfih2w9ue37x9K3TqZi/Ig2K\nNXDL/pRKaRLqaJ4M5Ij2iv0+VdOkkPymbZnGXR/fRcmxJWk8tTHng87HO/qncI7CLHxsIT2r9qTR\n1EasOroqyccPDQ9l+L/DGdJsSJL3pVRKFW9NQUQ+SM5AfM3evdC0qbejSDuOXT7Gm0veZOszW7n3\n7ntd2sYYw8sNXqZCvgp0mdOFsW3H0rNqzzuOYca2GZTPV56GxRve8T6USul0QHU89u61N64pzxMR\nnv3jWV6s96LLCSG6duXbsaz3MjrO7khIeAh9avRJ1PbHrxxn2Iph/LTnJ/587M9EH1+p1OS2E+L5\nMk+NPgoKsnMeXb2q9yUkh5nbZvLZms/Y2G9jkjqNN57cSJc5XTj44kEypc/kdJ3wiHAGLRlEUGgQ\nGdJl4ErIFX4L+I1+tfrxRsM3yJc13x0fX6mUIqkT4qVKly7Z4abO7N8PZcpoQkgOZ66d4fXFrzO1\n09QkjyKqU6QOlQtU5ttt38a7zq7/djF311wqF6hMmTxlqFmoJjsH7OTTVp9qQlCKRDQfGWMaAEOx\nj+IcKyK/eiooT9q3D774AmbPttNir1gBZWM9L107mZPHtZBrdJ3blX61+lG7SG237PPdJu/Sd35f\n+tTo43S6iXXH19G8dHOeq6ePz1PKmYTuaC4Uq+hVoAv2juZhngzKE27ehKefhsaN7UNydu2C99+3\nz1s+dizmupoUPO9G6A0e+P4BKuWvxIfNP3TbfpuWbErhHIWZt2ue0+XrT6ynXhF9TrJS8Umo+egr\nY8z7xpi7HO8vAd2wieGKxyNzoxMnoFkz20dw6BAMGwZFikD//vD88zYxnDlza33tZPas4LBguszp\nQpEcRfi649dxnmaWVO82eZfhK4cTIRFxlq0/uZ56RTUpKBWfeP8aReRBYAvwuzHmCeBlIDOQD0h4\nLmEf8s8/9nGanTvDDz/YJqPoXnsNevaE++6zTUrh4bfuZlae8drfr5E9U3ZmPDgjwecU3Kk2ZduQ\nOX1mFgTEvAH/esh19p/fT7WC1dx+TKVSC1cex5keGAh0BD4WkX+SIzBXxDf6KCQE5s2DL7+Ekyft\nvw88kPC+li6FwYPhyhU4eNDWHHLm9FDgadjNsJsUGW1nFC2as6jHjjNn5xy+2fINix9fHFX275F/\neX3x66x7ep3HjqtUSnBHo4+MMZ2MMcuBRcBOoAfQ2Rjzg+NJbD5p/Xr7zINp02DQIAgMvH1CAGjZ\n0j52c9QoGDhQE4KnLNy/kGoFq3k0IQA8UOEB1p9Yz7kb56LK1p9YT/2i9T16XKVSuoQacz8C2mEf\nhTlCRC6JyGvAYODj5AguMURg0iTo2NGOLlqyBB58ENInonXCGGjXDkaP9lycad33O7/n0SqPevw4\nWTNmpU3ZNvy699YgOe1PUOr2EkoKl4GuwEPA2chCEdkvIo94OrDECA+Hfv1sMli50vYfKN9zNfgq\nfx38i26Vk+dB1w9Xfph5u2+NQlp/QpOCUreTUFLogu1UzgDc+YQySWCMaWuM2WuM2WeMGRTfeoMH\n236Adev0cZm+bH7AfBqXaJxsN4m1L9+etcfXcv7Gec5eP8ulm5col7dcshxbqZQqoQnxzgHjkzGW\nGIwx6YAvgZbASWCDMWa+iOyNvt5PP9lRQxs2QPbs3ohUuSq5mo4iZcuUjdZlWjM/YD53Z7ubukXq\nun34q1KpjS//hdQD9ovIEREJBX4A4jQMPfusTQwFCiR7fCoRzt84z8qjK+lcIXnb9iKbkLTpSCnX\n+HJSKApEv9f4uKMshs8+g9rumSFBedCPu3+kTdk25MicvI/i6HBPB1YfW81fB//SpKCUC1L81NmH\nDg1l6FD7s5+fH35+ft4MRzmERYTx4sIXOXzpMDdCb7D7v91MfmBysseRPVN2WpVpxc97fqZukbrJ\nfnylfIG/vz/+/v4ureuzU2dHTsAnIm0d798CRERGRFvHI1Nnq6T7Yu0X/Lz3ZwY1GkTWjFnJfVdu\nqhes7vQpap72/Y7veXPJmxx75djtV1YqDUjo5jVfTgrpgQBsR/MpYD3wqIjsibaOy0nhRugNBi8b\nTLNSzehUoZMnQk4VRCTJJ+4z185QZWIV/unzD5UKeH8SqdDwUHae3UnNwjW9HYpSPiFFPk9BRMKB\n54G/gV3AD9ETQmJsOrmJWl/XYsmhJUzbOs2dYaYaIsLULVPJPyo/gRcDk7SvQUsG0ad6H59ICAAZ\n02fUhKCUi3y6T0FEFgEVkrKPrzd+zeDlg/mi7Re0KN2CihMqEh4R7pGJ2FKqi0EXefaPZ9n9325q\nF67NH/v+4IX6L9zRvlYfW82SwCXsee6O8rdSyst8tqbgDudvnOftpW+z+qnVPFr1UQpmL0ixnMXY\ndGqTt0PzGVeDr1JrUi0KZivI+qfX0792f/48cGfPKQ6PCOf5P59nVOtRyT7KSCnlHqk6KYxfP56u\nlbrGuIu1VelWLAlc4sWofMvcXXOpUagG49qNI0vGLLQu05qVR1dyI/RGove1/sR6gsODeaSKT82C\nopRKhBSfFFYeXem0/FrINSZsmMCbjd6MUd6qjCaF6KZunUrfGn2j3ue6Kxe1CtfC/7B/ove17NAy\n2pRt45URRkop90jxSeGhuQ+x8+zOOOWTNk2ieanm3JMv5mRITUs2ZcPJDXd0JZza7D23l8CLgbQr\n3y5Gefty7flzf+KbkJYeWkrL0i3dFZ5SygtSfFL4vM3ntJvVjiOXjkSVBYcFM2bNGN5u/Hac9XNk\nzkH1gtVZdXRVcobpk6Zvnc7j1R6P84D7duXbsfDAQhIzXDkoNIj1J9bTpGQTd4eplEpGPj36yBU9\nq/bkv+v/4TfDj741+tK6bGu2nd5GlburxDsMsWXpliwJXELrsq2TOVrfERYRxrfbvmXpE0vjLKt6\nd1WCw4LZd34fFfK7Nvhr1bFVVCtYjZyZ9elESqVkKb6mAPBSg5eY0mkKl25eov+C/gz4YwDvNHkn\n3vVblWnF0kNxT4ZpyaIDiyiZu6TTewmMMbQrZ2sLzhy+dJiNJzfGKFt2aJk2HSmVCqSKpADQonQL\nRrcZzfYB27ny9hWalmwa77r1i9Vn3/l9nL9xPs4yEeH0tdOeDDXZREgEERLhdNm0rdNidDDH1r58\n/P0Kw1YM4+F5DxMaHhpVtvTQUlqW0aSgVEqXapJCdNkzJfxghUzpM9G4RGOWH14eo/xq8FV6/dKL\nMl+UISg0yJMhetypq6do8E0DBvw+IM6yM9fOsDRwKT2q9Ih3+5ZlWrLm+BquhVyLUR4aHsqvAb+S\n5648TN86HYBLNy+x+7/dNCjWwK2fQSmV/FJlUnBF23JteX/5+4xdO5ZDFw+x9fRWak+qTdYMWalU\noBJrjq/xdoh3bMeZHTSY0oA2Zdvwx/4/+OfIP1HLRIQBfwygf+3+Cbb/58yckwbFGrAgYEGM8qWH\nllIhXwUmtJ/AR/9+REh4CCsOr6BBsQbcleEuj30mpVTySLNJYWDdgXza6lN2nNlB/W/q03xGc4b6\nDWVyp8m0KduG5YeW334nPmhJ4BJaftuST1t+yrAWwxjXbhz9F/QnOCwYgG82f0PgxUCGNR922309\nX/d5Pl/7eYxRSHN3zeXhyg9zX/H7qFygMlO3TNX+BKVSExFJsS8bftKFhYfJlZtXot7/deAvaTy1\nsVv2nZwiIiKk7BdlZeH+hTHKOn3fST7w/0D2/rdX8o3IJ7vO7nJpf+ER4VJ+XHn55/A/IiISHBYs\neUfklaOXjoqIyLrj66TYmGJSYXwFWXd8nfs/kFLKIxznTqfn1TRbU4gufbr0MebqaVS8EVtObUlx\nN7itOraKzBky06Zsm6gyYwxftvuScevG0XVuVz5s/iGVC1R2aX/pTDpeafAKn635DIClgUupmL8i\nxXMVB6Be0XpUL1id09dOU7uwPv5OqdRAk4IT2TJlo0ahGinuBrfpW6fTu3rvONNMFM9VnOEth1Ot\nYDUG1Inb8ZyQ3jV6s+bYGvad38fc3bbpKLoRrUbwXtP3dNZZpVIJn33Ijis8+eS1wcsGEy7hDG85\n3CP7d7cboTcoNqYYOwfupEiOIm7d9+Blgzl59SS/7P2F7QO2UyxnMbfuXymVvFLkQ3a8rXnp5nGG\nrPqyX/f+Sv1i9d2eEACeq/ccs3bMonKBypoQlErlNCnE475i97HjzA6uBl/1dihOrTm2JsaNaZFN\nR55QKHshXqj3Av1r9/fI/pVSvkOTQjyyZMxCnSJ14p2a25t+2fMLjaY2osucLly6eYnjV46z6dQm\nOlfo7LFjjrp/FE9Uf8Jj+1dK+QZNCgloXsr3mpCuBF/hxUUv8vfjf1MyV0nqTKrD4OWD6VapG1ky\nZvF2eEqpFM5rScEYM9IYs8cYs9UY85MxJme0ZW8bY/Y7lt/vrRj9Svn5XFJ4Z+k7tC3bllZlWjGu\n3TiGNR/Gj7t/pG/N+OcxUkopV3lt9JExphWwTEQijDGfYm+meNsYUxmYBdQFigFLgPLOhhl5cvQR\n2Ocy5BuZj5OvnfSJKaHXHFvDQ3MfYtfAXeTJkieqPCQ8hEzpM3kxMqVUSuKTo49EZIlIVE/pWmwC\nAOgE/CAiYSJyGNgP1PNCiGTOkJkK+Suw99xebxw+hpDwEPr/3p/P23weIyEAmhCUUm7jK30KfYHI\neZqLAseiLTvhKPOKivkr+kRSGLt2LMVyFqP7vd29HYpSKhXz6JPXjDGLgYLRiwAB3hWRBY513gVC\nReT7OznG0KFDo3728/PDz8/vTsN1qmI+7yeFY5ePMXLVSNY9vS7O3cpKKXU7/v7++Pv7u7SuV+9o\nNsb0AfoBLUQk2FH2FrZ/YYTj/SJgiIisc7K9R/sUAObtmsf3O7/n5x4/e/Q4Cek2txtV7q7CUL+h\nXotBKZV6+GSfgjGmLfAG0CkyITj8BjxijMlkjCkNlAPWeyNGgAr5KxBwPsBbh2fRgUVsOb2FQY0G\neS0GpVTa4dHmo9sYD2QCFjuaRNaKyEAR2W2MmQvsBkKBgR6vDiSgfN7yBF4MJCwijAzpkvfruhl2\nk+f/fJ7x7cbrPQhKqWThtaQgIuUTWPYJ8EkyhhOvLBmzUDh7YQ5dPET5fPGG7BGzts/innz30L58\n+2Q9rlIq7fKV0Uc+zVsjkH4N+JVe1Xol+3GVUmmXJgUXeCMpXA+5zorDK2hXrl2yHlcplbZpUnBB\nhXzJ39m8OHAxdYvWjXOjmlJKeZImBRd4o6bwW8BvHp31VCmlnNGk4ILkTgrhEeH8vu93OlXolGzH\nVEop0KTgkruz3U24hHPuxrlkOd6a42sonKMwpXKXSpbjKaVUJE0KLjDGUDF/RQLOJU+/gjYdKaW8\nRZOCiyrkS77ZUucHzNekoJTyCk0KLkqufoW95/ZyPeQ6tQrX8vixlFIqNm9Oc5GiVMxfkSlbpnhk\n399s/obZO2ZzLeQap6+d5qFKD+lsqEopr9Ck4CJP1RRCwkN4e+nbfN3xa4rmKEq2TNm4J989bj+O\nUkq5QpOCi8rkKcOxy8cIDgsmc4bMbtvvgoAFVLm7Cl0rdXXbPpVS6k5pn4KLMqXPRMncJTlw4YBb\n9ztt6zSerPGkW/eplFJ3SpNCIjQp0YQF+xa4bX+nrp5i9bHVPFTpIbftUymlkkKTQiIMqDOAiRsn\nEh4R7pb9zdw+k66VupItUza37E8ppZJKk0Ii1C5SmyI5ivD7vt+TvC8RYeqWqdp0pJTyKZoUEum5\nupK6RxcAAAwESURBVM8xYcOEJO9n7fG1REgEDYs3dENUSinlHpoUEunhyg+z7cy2GFNebDq5ifUn\nEvcY6Wlbp9GnRh+9H0Ep5VO8nhSMMa8ZYyKMMXmjlb1tjNlvjNljjLnfm/HFljlDZp6u+TT/2/A/\nAGZum0nT6U15Z+k7Lu9DRFiwbwE97u3hqTCVUuqOePU+BWNMMaA1cCRaWSWgO1AJKAYsMcaUFxHx\nTpRxPVPnGWp+XZMM6TLwa8CvLHtiGa1mtuJayDWyZ8p+2+0PXTqEwVAmT5lkiFYppVzn7ZrC58Ab\nsco6Az+ISJiIHAb2A/WSO7CElMhVgpalW7Lp1CbWPb2O+sXqU7dIXZYfWu7S9quOrqJh8YbadKSU\n8jleqykYYzoBx0RkR6yTY1FgTbT3JxxlPuW7rt+RIV0G0hmbV9uWa8uiA4t4oMIDt9129bHVNCre\nyNMhKqVUonk0KRhjFgMFoxcBArwHvINtOkqRMqXPFON9u3Lt6PxDZ0TktjWAVcdW8WRNHYqqlPI9\nHk0KIuL0pG+MqQKUArYZewYtBmw2xtTD1gxKRFu9mKPMqaFDh0b97Ofnh5+fX1LDviNV7q5CSHgI\n+y/sT3BCu0s3LxF4MZCahWomY3RKqbTM398ff39/l9Y1vtB/a4w5BNQSkYvGmMrALKA+ttloMeC0\no9kY40v9zzz929NUvbsqLzV4Kd51Fh1YxKcrP8W/j3/yBaaUUtEYYxARp00a3u5ojiTYpiVEZDcw\nF9gN/AkM9KkzfwLalWvHooOLElxn1dFV2p+glPJZPpEURKSMiFyI9v4TESknIpVE5G9vxpYYrcq0\nYtXRVQSFBsW7zqpjq2hUQpOCUso3+URSSC1y3ZWLGoVqsOLICqfLQ8ND2XByA/cVuy+ZI1NKKddo\nUnCzduXasXD/QqfLtp3ZRslcJcmTJU8yR6WUUq7RpOBmLcu0jLemoP0JSilfp0nBzaoXrM7+C/u5\nEXojzrLVx1drf4JSyqdpUnCzzBkyU7lAZbae3hqjXES0pqCU8nmaFDygbpG6cabSPnjxIILoJHhK\nKZ+mScED6hapy4aTG2KU+R/2x6+Un06Cp5TyaZoUPKBu0bpsOBEzKSw/vJzmpZp7KSKllHKNJgUP\nqJS/EqeuneLSzUuA7U9YfkiTglLK92lS8ID06dJTs1BNNp7cCMC+8/vIkC6D9icopXyeJgUPid7Z\nrP0JSqmUQpOCh9QtequzWfsTlFIphSYFD6lbxHY2iwj+h/1pXlqTglLK92lS8JAyecoQFBbE8sPL\nyZIxC6Vyl/J2SEopdVuaFDzEGEPdInUZtXoUfqX8vB2OUkq5RJOCB9UtUpdFBxZpf4JSKsXQpOBB\ndYvWBdCaglIqxdCk4EENizekx709KJGrhLdDUUopl5gU8vhjp4wxKeXxzUop5TOMMYiI0xunvFpT\nMMa8YIzZY4zZYYz5NFr528aY/Y5l93szRqWUSku8lhSMMX7AA0BVEakKfOYorwR0ByoB7YD/mRR2\nK7C/v7+3Q3BK40ocjStxNK7E8dW4vFlTGAB8KiJhACJyzlHeGfhBRMJE5DCwH6jnnRDvjK/+sjWu\nxNG4EkfjShxfjcubSeEeoKkxZq0xZrkxprajvChwLNp6Jxxld8yVL9/VX5Ar6/2/vXOPsaOq4/jn\nS7fVQi10tbSGlZc8GqJSqGARCNEgYjQoiSgQUoxaEhuVqAmoMdZoA9ZEaZGIGAXiA2PBCKuWRxVS\n1MQabOsWSgtGSgOhRTBabIw8+vOPc3YY1r30PubeGZrvJ5ns3DNnzn7u2bvzu3Nes23btsrKsldn\nZVXp3o5XlfVgr87y2auz39duvr4GBUlrJI2Vtk355znAEDArIhYClwE398tjX7/ItZvPXu3ngX3/\nYtJuPnt1lu+VHhRqG30kaTWwPCLW5tcPAwuBxQAR8fWcfgewNCLWTVKGhx4ZY0wXtBp9NDRokRK3\nAu8E1ko6BpgWEU9LGgV+IulbpGajo4A/TVZAqzdljDGmO+oMCjcA10vaBPwXWAQQEZslrQI2A88B\nSzwZwRhjBsMrevKaMcaYamnUMheSRiTdLemB3Cn96Zw+S9JdkrZKulPSgaVzJp3oJumC3Km9UdJq\nScMN8fqwpL/kcq7s1qkbL0nDOf8zkq6eUNaJub4ekrSiQV7LJG2XtKsXpyq9JE2X9KvSxMsrmuCV\nj90uaUMup6c5PlV6lcoclTTWrVPVXkojH7fkOlsv6XUN8Zoq6bp8zmZJ53br1TER0ZgNmAvMz/sz\ngK3APGA5cFlOv5w0vwHgOGADqRnscOCvgIApwE7S6Cby+V9ugNcw8CgwnPPdALxjgF77A28HLgGu\nnlDWOuCkvL8aeHdDvE4G5gC7avh8TeoFTAfOyPtDwL0Nqq8Zpf1bgA81wSsfPxf4MTDWhL9jPnYP\ncEKvn60+eH0F+Grp9XAVjm29j0H9oi4r+VbgTGALMKdU8Vvy/ueBy0v5bwfelv9RdwKHki7G1wIf\nb4DXW4E1pfSLgGsG5VXKdzEvvcjNBTaXXp8PXFu314RjPQeFfnjl4yuAjzXJC5gKjALnNcELOIAU\nPOfRY1Co2OseYEHVn60KvLYD0/vhtbetUc1HZSQdDswH/kiq0J0AEbEDODhnm3SiW6RZ0kuATcBj\npCUzflC3F+mO4VhJh0oaAj4AvGGAXq04hFRP4zxGjxMGK/LqG1V5STqItFzLb5vipTSMewewi3S3\n0ASvr5GWsvlPFT4VegHcmJuOvtQEL73YDL1M0p8l/UzS7Krc9kYjg4KkGaQP86UR8W9gYm/4y/aO\n5wvuJ4DjI+IQUnD4Yt1eEfHP7LUKWAs8ArxQt1e/2Ne9JE0BbgJWRFqSpRFeEXE28HrgVaRh37V6\nSToeeGNEjJLu3CsZSl5RfV0Yae2104HTJV3UAK8hYAT4fUQsIAWWb/bq1S6NCwr5gn4L8KOIuC0n\n75Q0Jx+fCzyZ0x/npd+0R3LafCBK/6irgFMa4EVE/DoiFkbEqcBDeRuUVyta+tbsVTkVe30P2BoR\n326YFxHxLKn56P0N8DoFWCDpb8DvgGMk3d0ALyLiifxzNynA97TOWhVeEfE0sDsifpGTbgZO6MWr\nExoXFIDrSe3bK0tpo8BH8v7FwG2l9PMlTZN0BC9OdHscOE7Sa3O+dwEPNsCL8dtASbNITVzfH6BX\nmeLbWr6l/Zekk/NolUUtzhmoV5vptXhJWgbMjIjPNMVL0gH5ojN+cXovqT27Vq+I+G5EjETEkcBp\npEDa6x1MFfU1ZfwaIWkq8D7g/rq9Mr+UNP4c3zNJ87YGQx0dGa024FRSc8pG0uid9cDZpFE7vyH1\n5t8FHFQ65wuktvoHgbNK6ZfkitxI+iPMaojXTcADpA9fT52AXXo9AjxFam/eDszL6QtIzWwPAysb\n5LWc1D/zfE7vZRRZJV6k/pY9+e84Xs5HG+B1MOnLx0ZgDFgJ7Fe314QyD6P30UdV1df+wH25nE3A\nVeS5W3XXF2mQzNpc1hpgpJc662Tz5DVjjDEFTWw+MsYYUxMOCsYYYwocFIwxxhQ4KBhjjClwUDDG\nGFPgoGCMMabAQcGYDpD0Ql4n5/683PJn84S/lzvnMEkXDMrRmF5wUDCmM3ZHxIkR8SbSTPn3AEv3\ncs4RwIV9NzOmAhwUjOmSiHiKNHP+k1DcEdwr6b68LcxZrwROy3cYl0raT9I3JK1TegjU4rregzET\n8YxmYzpA0q6ImDkh7R/AscAzwJ6IeFbSUcBPI+IkSWcAn4uIc3L+xcDsiLhC0jTgD8AHI+LRwb4b\nY/6foboFjNkHGO9TmAZcI2k+aQ2co1vkPwt4s6Tz8uuZOa+DgqkdBwVjekDSkcDzEfF3SUuBHRHx\nlvyshVYPlBHwqYhYMzBRY9rEfQrGdEZ56eXZpEe9jj9P4UDgiby/iPSscEjNSq8plXEnsCQvb42k\noyVN76e0Me3iOwVjOuPVktaTmoqeA34YEVflY98Bfi5pEXAHsDunjwF7JG0AboyIlflxjevzcNYn\nSY9mNaZ23NFsjDGmwM1HxhhjChwUjDHGFDgoGGOMKXBQMMYYU+CgYIwxpsBBwRhjTIGDgjHGmAIH\nBWOMMQX/A3uI0XlBM3TJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11e97940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_data[\"pnl\"] = test_data[\"Adj Close\"].diff().shift(-1).fillna(0)*test_data[\"positions\"]/np.where(test_data[\"Adj Close\"]!=0,test_data[\"Adj Close\"],1)\n",
    "test_data[\"perc\"] = (test_data[\"Adj Close\"] - test_data[\"Adj Close\"].shift(1)) / test_data[\"Adj Close\"].shift(1)\n",
    "monthly = test_data.pnl.resample(\"M\").sum()\n",
    "monthly_spy = test_data[\"perc\"].resample(\"M\").sum()\n",
    "avg_return = np.mean(monthly)\n",
    "std_return = np.std(monthly)\n",
    "sharpe = np.sqrt(12) * avg_return / std_return\n",
    "drawdown = create_drawdowns(monthly.cumsum())\n",
    "spy_drawdown = create_drawdowns(monthly_spy.cumsum())\n",
    "print(\"TRADING STATS\")\n",
    "print(\"AVG Monthly Return :: \" + \"{0:.2f}\".format(round(avg_return*100,2))+ \"%\")\n",
    "print(\"STD Monthly        :: \" + \"{0:.2f}\".format(round(std_return*100,2))+ \"%\")\n",
    "print(\"SHARPE             :: \" + \"{0:.2f}\".format(round(sharpe,2)))\n",
    "print(\"MAX DRAWDOWN       :: \" + \"{0:.2f}\".format(round(drawdown[0]*100,2)) + \"%, \" + str(drawdown[1]) + \" months\" )\n",
    "print(\"Correlation to SPY :: \" + \"{0:.2f}\".format(round(np.corrcoef(test_data[\"pnl\"], test_data[\"diff\"])[0][1],2)))\n",
    "print(\"NUMBER OF TRADES   :: \" + str(np.sum(test_data.positions.abs())))\n",
    "print(\"TOTAL TRADING DAYS :: \" + str(len(data)))\n",
    "print(\"SPY MONTHLY RETURN :: \" + \"{0:.2f}\".format(round(monthly_spy.mean()*100,2)) + \"%\")\n",
    "print(\"SPY STD RETURN     :: \" + \"{0:.2f}\".format(round(monthly_spy.std()*100,2)) + \"%\")\n",
    "print(\"SPY SHARPE         :: \" + \"{0:.2f}\".format(round(monthly_spy.mean()/monthly_spy.std()*np.sqrt(12),2)))\n",
    "print(\"SPY DRAWDOWN       :: \" + \"{0:.2f}\".format(round(spy_drawdown[0]*100,2)) + \"%, \"  + str(spy_drawdown[1]) + \" months\" )\n",
    "\n",
    "print(drawdown[0])\n",
    "(monthly.cumsum()*100).plot()\n",
    "(monthly_spy.cumsum()*100).plot()\n",
    "plt.legend([\"NN\", \"SPY\"],loc=2)\n",
    "plt.ylabel(\"% Return\")\n",
    "plt.title(\"TRADING SPY OUT OF SAMPLE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot shows the % returns when we trade using SPY and NN based models only when we are > 55% sure of the predicted directionality.\n",
    "\n",
    "It is interesting to see here how the NN performs during the [financial crisis](https://en.wikipedia.org/wiki/Financial_crisis_of_2007%E2%80%932008). This strategy makes money when many are losing ( refer to the dip in the green line in the plot between 2008-2010)! More importantly, this strategy has a significantly lower volatility (lower standard deviation in the blue line relative to the green line). \n",
    "\n",
    "In the summary statistic, you can also see a lower maximum drawdown of 13.62% for the NN model compared to the 55.37% for the SPY index over the same period of time. Similarly, a higher Sharpe ratio close to 1 from the NN model compared to 0.6 from the SPY index for the same period indicates the NN based decisions to be more stable (less risky). Thus, from a quant perspective it is more likely to be profitable (it makes money in many different market time periods). More importantly, the NN strategy is relatively uncorrelated with the SPY index (only 0.06 close to 0). \n",
    "\n",
    "**Suggested task**: Try to experiment with this network with different stocks and see what happens to the Sharpe ratio when you have multiple trading streams that are uncorrelated with each other. \n",
    "\n",
    "Remember when trading you can trade with leverage, and increase your return profile (as long as you are willing to handle the down times). The best trading strategies are the ones that can consistently make money no matter which market they are trading and no matter what is happening in the market. This \"edge\" is termed alpha in the quant industry.\n",
    "\n",
    "It is important to notice how much of a drawdown you will experience when trading (http://www.investopedia.com/terms/m/maximum-drawdown-mdd.asp?lgl=no-infinite). Since our current model is calculating only the percentage returns where returns are not reinvested, and we are not holding but trading, the larger the drawdown the harder it is to recover (if you lost 90% of your portfolio then have a 90% gain, you will only be at 19% of your starting value). \n",
    "\n",
    "We can only predict on close prices and not the adjusted close prices since the adjusted close prices incorporate data from the future (such as stock splits and dividends) which can incorporate bias into our network. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRADING STATS\n",
      "AVG Monthly Return :: 1.10%\n",
      "STD Monthly        :: 7.11%\n",
      "SHARPE             :: 0.54\n",
      "MAX DRAWDOWN       :: 30.30%, 33.0 months\n",
      "Correlation to SPY :: 0.04\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEZCAYAAABmTgnDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmcTeUfwPHP19iXrNmLhFCyJluWaJUUCWUt6qfS8mtR\n/VoUJa2SKBIlFEokZN8JIfsSMfYtjIbBzHx/fzx3GLMvdxvzfb9e9+Xeszzne84d53vP8zznOaKq\nGGOMMUnJEugAjDHGBD9LFsYYY5JlycIYY0yyLFkYY4xJliULY4wxybJkYYwxJlmWLIwxxiTLkoW5\nQEROiUiY5xUlIqdjTesgIm+KyDnP539EZLGI1E2gnLKe9T9PYF50rDKPiMgsEXkwzjLzROQRz/vG\nnnUGx1lmkYh0jvW5uIgME5F9nrL/EpGvRaRiEvv7qojs9CwfKiLjYs2bLyJnPPMOi8hEzzYeFZFN\nIpIt1rKFReSQiNyeyHZKich3InLUs+/LRaRFrPllPPuYJc56I0Wkr4i8EuuYnRGRSM/7UyKyPpFt\nZheR/iKyW0TCRWSriLwQZ5nY+xhT/s2JlPeoiGwWkZMickBEpopInjjL9PHsx01xpnfxTP8ozvRW\nnulfxzkOMX+DO0Wkd6zlo0WkXAKxdYl1TGLvS/GE9sWkjSULc4Gq5lPVK1T1CmA30CLWtJgT6fee\n+UWA+cCEBIrqDPwDtIt9Uo3ZDHCjp4zrgG+AwSLyehKhhQOdROTqhGaKSCFgKZALaOApuyawALgt\nkXW6AA8Dt3qWrw3MiRPnE555FYGCwMeqOgLYC7wZa9mBwFRVnZnAdgoCi4EIoDLuuA0ExopI6zjb\nS4iqav+Y7wH4D7DU853kU9Wqiaw3EWgK3AnkAzoBj4nIpwntY6zv+fcE9qEx8A7QTlXze/bjhwS2\n2Qk4hvv+49oBPBgnIXYGtsbdXyC/Z18fAt6IlYSTuoM45pjE3peDSSxvUsmShUmMeF4JUtVoYAxQ\nUkQKx5ndGXgNOA+0TKxcVf1HVb8DegKvek6sCTkBjAL6JDL/v8BJVe2kqrs8ZYep6jeqGu/qxqM2\n8Fus5Q+r6lcJxIqqngB+BG7wTO8B9BSRG0XkDtxJ+b9JxHZKVbur6hFVPauq3+NOvh8nsk66iEgz\noDnQWlU3q2q0qq4AOgJPxvl1nuh3HEtt3Ml4HbjjoaqjVTU81jYbAcWBp4EOIpI1ThkHgfXAHZ7l\nCwL1gSkJ7YJnO8uBjVw87imJ1fiIJQuTJiKSHeiC+yV5PNb0W4BSwPe4q44uKShuMpAVqJPIfMWd\nXNuISIUE5jcDJqU4eGc50FlEXhCRWnGrgGITkSJAG2A1gKruBt4ARgJDgJ6qejKR1ZvjEk1c44Gr\nE9mf9GoO/K6q+2NP9CSMvbjjlRq/A3d4qpnqe777uDoDv3DxSjPujwQFvuXi30N74GfgXAJlCYCI\nNACq4DnuJrAsWZjUaici/wCngUeBBzxXGTE6A9M8J8+xwJ2ek22iVDUSOAoUSmKZw8AXwNsJzC6C\n++UKgIi0FJHjnnrrGYmUNwboBdyOq047JCIvxVnsM8++rgH2Ac/HWv9z3JXTWlX9JYndKwIcSGD6\ngVjzvS2xbcZsN/Y2B4lrfzouIqsSWkFVFwOtgRrAVOCoiHwkIjEn9VxAW2CM57ucSMJVUT8DjUXk\nCs/8bxNYRoAjInIMGAb0VtX5Se6tU8+zHzH7sj0F65hUsGRhUusHVS0EFAU24KooABCRnLiTxli4\nUI2wB1f3nChPlcWVuKuUpAzA/cK9Mc70Y0CJmA+q+ouqFgSeAxL6FRyz3DhVvR0ogGsL6Csisds4\neqlqIVW9SlU7q2rc+DbjqkmScjR2bLGUiDU/0vM+bvtONlxCSq3Ethmz3aOxPj/t2ceCqlo7kXVQ\n1d9UtZXnu28FdAW6e2a39sQ53fN5LHB33OpJVY0AfsVVURZS1WUJbQoorKqFVfX6JKoR41rm2Y+Y\nffHFFVumZsnCpImq/gM8DvQRkWKeya2BK4Ahnh4zB4CSJF8VdR/uZLMyBdscCPTl0sbOOZ4y0kRV\no1T1R2AdF+vHvWU27rjE1Q4IVdXtuF/754GycZa5BtfRIC3bvFlESsWe6OnpVJpLG/JTTVXnAXO5\neKy6AHmBUM93Ph5XrZjQj4TRuHac0UlswtomgpAlC5NmqroNmAHEdG/sAowAqgLVPK+GQHURuT7u\n+iJSUEQeBgYD76nq8bjLJOATXMNo5VjTPgYKisjomMZbEckHVE+sEE93y7tFJK84d+Hqx5enIIbU\n+ATILyIjRKSYiOQQkQ7AK8ALcKGzwI/AOyJSSESyepapzMVf6ymmqnNwCeFHEakiIlnEdXEeDQxR\n1Z2pKU9E7hWRdiJSwPO5DtAYWCYiJYFbgRa4410NuBF4nwR+JKhqTA+1wXHnxWwumXByeI5hzCvm\nHGYJxscsWZjEpPRBJx8CPTzdWpsCn3h6FsW8VuNOeLFPHH+KSBiwHXgEeEZV30rJtlX1FO5EVCjW\ntGNAXVz31MWeslfjfu32TKSoMOBV3C/348B7wH9iVY2kZP+TXcZzNdQQ1613E64K6Fmgo6pOjLXo\nE7juxuuAQ57Pd6vqkRTEkZA2wDxcMj+Fax8YrqpPpyZ+j+O4HmDbROSkp6wBnl5dnYDVqjon9vcO\nDAKqikiVuIWp6jxPD7OEJBWT4qo+TwNnPP929cyrm8B9FrVSuH8mBSTQDz8SkRHAPcAhVb3RM+1N\n3B/nYc9ir6rqDM+8V3AnmEjcSSZe33ZjjDHeFQzJoiHwL/BtnGRxSlU/jrNsZVzj2U24utfZQAUN\n9E4YY8xlLuDVUJ5ueQnVVSdUB9kKdwdxpOdmqu0k3jffGGOMlwQ8WSThKRFZKyJfiUh+z7RSuK6Y\nMfZ5phljjPGhYE0WQ4Byqlodd7PVR8ksb4wxxofijt8SFOL0ABmOG0YA3JXEVbHmlfZMi0dErB3D\nGGPSQFXjNQMEy5XFJYPWxRlauDWuuxy4Qcfaixt++RqgPLAisUJVNcnXm2++mewyKV0upWU1btz4\nso7Lm7GnJCaLy+LyZVwZ+f9iWuNKTMCvLERkLNAEKCwiobihn5uKSHUgGtiFu1MYVd0kIuNx/dXP\n44ZXTvMVRJMmTby2XErLKlu2rNfKCsa4vBl7SmJKaVkWV+rKsri8v72Mfo5IUXbKiC+3a8HnzTff\nDHQICQrGuIIxJlWLK7UsrtQJdFyec2e8c2qwVENlGinO4n4WjHEFY0xgcaWWxZU6wRpXwG/K8xUR\n0ct134wxxldEBE2ggTvgbRb+VrZsWXbvTstAnpePMmXKsGvXrkCHYYzJQDLdlYUnawYgouBhx8AY\nk5jEriyszcIYY0yyLFkYY4xJliULY4wxybJkYYwxJlmWLIJI2bJlKVasGGfOnLkwbcSIEdx6660A\nZMmShWrVql2yzuuvv84jjzzi1ziNMZmPJYsgIiJER0czcODARJfZv38/33//vR+jMsYYSxZB58UX\nX+Sjjz4iLCwswfkvvfQSb7zxBtHR0X6OzBiTkLAwOH060FH4niWLIFO7dm2aNGnCBx98kOD81q1b\nkz9/fkaNGuXfwIzJBHbtgk6dYOxYOHUq+eUPHYKbboJevXweWsBZsohDxDuv9HjrrbcYPHgwx44d\nSyA+4e2336Zv375ERkamb0PGmAtU4bHH3P/fMWOgdGlo0QJ69ID//hf69oW//rq4/D//wO23u2Um\nTYJ9CT5Z5/KR6Yb7SE4w3Nh8/fXXc88999C/f38qV64cb/5dd91F6dKl+eKLLwIQnTGXp2+/haNH\nYdo0yJoVjh+HuXPh2DF3lbF3L9StC61awbPPuiTSrBl89BFER8PAgZBIhcBlwZJFkOrTpw81a9bk\n+eefT3B+v3796NChAx06dPBzZMZkHD/9BFu3wrlzcP48dO4MFSvGX+7QIXjpJZg+3SUKgIIFoU2b\nS5d74w34+GOoXx8eesglChF35VGjBvzvf1CggO/3KxCsGipIXXvttbRr145BgwYlOL9x48bccMMN\nfPPNN36OzJiM4d134eWXXQN0dLS7OrjzTnf1ENczz0DXrlCzZtJlFizoqqOOH4cvv7xY5Xz11a46\nauhQr+9G0LAriyAicRo73njjDb777rsL0+PO79evH/Xq1Ys33ZjMrl8/+O47WLAASpS4OD13bmjd\nGmbPhuzZITIS3nsP/vgDRo5MeflZEzhzvvQS3HYbPPcc5MyZ/n0INjbqbCZkx8Bczvr2db2Z5s2D\n4sUvnRcd7aqWChZ07Q6PPOLeDx8OKXz6a5Luuce9/vOf9JcVKImNOmvJIhOyY2AuV9OmwRNPwO+/\nQ7FiCS/z77/QsCHs2eMapLt1S38Pxhh//AF33w2rVsFVV3mnTH+zZHFxeqY/UdoxMJej48ehalUY\nPRqaNk162X/+cVVQRYt6P47+/eG332DOHAgJcdMWLIDXXoMffoCSJb2/TW+y51kYYy5rzz4L99+f\nfKIAKFTIN4kCXNtFliyuLQRg4kRo2xauucbFF2votwzFriwyITsG5nIzZYrrvvrnn5AnT6CjcTfo\n1aoF7dvDhAkwdSpUrw4dOrjG8dGjvVf15W12ZWGMuSwdOQI9e7reTMGQKABKlYJhw2DhQli0yN2D\nIQJffw2bN2fMm/fsyiITsmNgLhfR0a5BuXr1i9U+wW7vXjee1MSJ0KCBd8s+eBDy54dcudK2/q5d\ncM01CV9Z2H0WxpgMq39/N+Jrv36BjiTlSpeGQYPcOFSrV0OOHClbLzwcli6F+fNh40bX1bdSJddg\nvmQJ/Pqrq/6KiHDtI9WquZsFCxRwrwYN4MYbEy//55+THhDRriwyITsG5nIwb54bcmPVKlftk5Go\nujGmatd2Q4iAu9O8fXtYs8Z1uy1dGrJlg/37XRI4dMjdYd6kiev1FRoKW7a4f+vWdVdYN93kenlt\n3uzabw4cgBMnXO+vX35xAx/26xe/W++yZXDvvW64k5tusq6zMdMz/YnSjoEJNtu2uXsj1q1zv5or\nVnTDZzRqlPAv75gT56hR7q7pjGjPHteWsXix6511110uebz2mksOe/e68axKlnTJsHTp9N0ZHhYG\n77/vhiTp2NEl2jp1YPt2d5y//tolnMQauFHVy/Lldi2+xKYHi0WLFmn9+vU1f/78WrhwYW3YsKGu\nWrVKR40apSEhIZovXz7Nnz+/1qhRQ6dOnaqHDh3SIkWK6IIFCy4pp1u3btqhQ4cEtxHsx8AbDhxQ\nnTkz0FGY5ERGqr7xhmrRoqrt26u++67qpEmqffuq1q2resUVqt9+G3+9hx5Sfekl/8frbYMGqdap\no1qhgjsO0dG+3+aePaqvv65aqZLq1Verli6tOnz4xfme80P8c2pCEy+HV0ZMFmFhYVqgQAH94Ycf\nNDo6WiMiInTWrFm6fv16HTVqlN5yyy0Xlv3ss880d+7ceuLECR09erRWqFBBIyIiVFV19uzZWqxY\nMT127FiC2wnmY5Aas2erVq2q+r//XTr9/HnVBg1Uc+dWHTEiMLGZ5O3bp9q4sWqzZi65J2TdOtXC\nhVW3b784bfZs1TJlVMPD/RGlb0VGqt5zj+pnn/l/29HR7vjOmHHpdEsWlx6IoLRq1SotWLBggvPi\nJovw8HAVEf3jjz9UVbVly5b60ksv6ZkzZ7R8+fI6fvz4RLcTzMcgJUJDVdu2VS1bVnX0aNVrr1Ud\nOfLi/JdfVr39dtXNm1VLlVL96quAhWriiI5WXbNG9cUX3dXE22+7E2ZSBg5UrVfP/QiIiFC97jrV\nyZP9E29mlFiysN5QQaRixYqEhITQtWtX2rdvT926dSmQwOD4kZGRDB8+nHz58lGhQgUAhg4dSrVq\n1di6dStVq1albdu2/g7fL06dcr06unaFb75xXQRr13Z1ruXKuR4jo0e7XiZFi7qH1zRr5tZ99NGA\nhp5pnTnj6uVnz3aNrGfOuPry+fMhgWd7xdOrl1vv/ffd54oVXWOs8S9r4I47/y3v3Fapb6btuG7d\nupUBAwYwe/ZsDhw4QIsWLRg2bBjTp0+ne/fu5MuXj6xZs1K+fHneeecdmsYa22DIkCH07t2bHTt2\nUDSJsQwycgP3Cy+45xHEfQT5rFnu2ckA48e75BFj82b3sJpjx9wwDMZ/vvrKDdldrRo0bw533OF6\n7qT27uU9e9wd0dHRrveTN0aINQmzgQQvTs8wJ8pt27bx8MMPU6FCBe644w5GjBjBwoULE11+wYIF\ndOrUidDQ0CTLzUjHILb1691VwoYNCY/rM3q063P/+OPx51WsCD/+6LocGv/YuNF181y8GK67Lv3l\nTZ3qEn6XLukvyyTOhvvIgCpWrEjXrl3ZuHFjoEMJuOhoN6TD228nPgBcp04JJwpwQ1IvXuy7+Myl\nIiLcOEgDBngnUYB7ToQlisCxNosgsnXrVn799VfatWtHqVKl2LNnD+PGjaNu3bqBDs0vVq1y1Uyn\nTrl67ZAQuPVW19/+77/dc5R79Ehb2Q0auPaLnj29G3Nm988/buyjxYvd3cQPPOCGm3j5ZXc1161b\noCM03mJXFkEkX758/P7779x8883ky5eP+vXrc+ONN/LRRx8FOjSfO3XK3b3atq17tvGECe4moaJF\noU8fePppdzNRzPMBUqthQzckgvGOf/5xQ4GXKQNDhkC+fG64iauvdlcAP/3kBtIL1pFVTeoFvM1C\nREYA9wCHVPVGz7SCwA9AGWAX8KCqnvTMewV4BIgEnlHVmYmUm+HbLHwlGI9B165uaIPhwxOef/68\nm59Wqi7xrF6dcZ9gFixOnXKN1Q0busH7Yn8vx465RF+rlht6wmQ8wdxmMRK4I860l4HZqnodMBd4\nBUBEqgAPApWBu4AhIvbbJaP7/ns3Ns3AgYkvk55EAe4XboMGdnWRXhERbkyjatXgww/jfy+FC7vn\nT1uiuPwEPFmo6mLgeJzJrYBvPO+/Ae7zvL8X+F5VI1V1F7AdqOOPOI1v7N7tqpjGjvX9swj8WRX1\nyy+wc6d/tuUvUVHw4IPu2dZDh1oVU2YT8GSRiKKqeghAVQ8CMf1fSgF7Yi23zzPNZFBvvOEanWvV\n8v22GjTwT4+oMWOge3d3P8Hkyb7fnr/07+8Go/v227S3HZmMK6P0hkpTBXufPn0uvG/SpAlNmjTx\nUjjGG3bvdr/Ad+zwz/Zq1nQjbIaFwRVX+GYbM2a4x3vOnQv//ut+iS9ZAu+8k/6qtEBauhQGD4Y/\n/sjY+2Himz9/PvPnz09+wYTGAPH3C9eQvS7W581AMc/74sBmz/uXgd6xlpsB3JxImUmNe5KpBcsx\nePppN0aQPzVqpPrbbwnPW71a9Z9/0l72smWqRYqoLllycdrRo6rNm6s+9VTay/W3efNUX33VDfSn\nqnr8uBuH6+efAxqW8RMSGRsqWKqhxPOKMQXo6nnfBZgca3p7EckuItcA5YEV/grSeM+RI+6O62ef\n9e92E6uKOnHCPRjmrbfSVu6ePXD//W68qvr1L04vXBh++ME9QvP339NWtj9Nn+66Lx8+DDfc4MZl\n6tbN3evSqlWgozMBlVAG8ecLGAvsB84CoUA3oCAwG9gKzAQKxFr+FeAv3NXH7UmUm2DWLFOmjOKq\ntTLtq0yZMun98ZFur7+u2qOH/7f766+qTZvGn/7ii26o6IIF3S/p1Dh7VvXmm1UHDEh8mTFjVG+8\nUfXcudSV7U+TJ6teeaXq0qXu88GD7rg0b656+nRgYzP+QyJXFgG/z8JXErvPwgTeqVNuhNhly6B8\nef9u+/hxd+PYn3+6GMD1WrrpJjfm1Isvum6hL76Y8jKfecY96H7SpMQHKlR1g+jdfru7Sz05e/Z4\n736Qc+cge/bE56vCiBHwv/+58Zes22vmZgMJmoB67TVYuRJy53Z3/xYv7qpnAmHoUNfg/NtvcP31\nrtqlWjUX4+rVrrpl586UNeSOH++GtvjjDyhYMOlld+yAm2+GmTPh7Fm3jXPnXBIpWdIts2mTK++X\nX2DcOHdXe3pMnerKmDrVDeoX19at7r6IsDBXhXbDDenbnsn47LGqfhIdrXr4cOrWee891S++8E08\nwWDmTNVrrlGdNk114kT3mMzUHiNvGz1atVgx1U8/Vb3qqkufutakierYscmX8eOP7iluq1alfLuf\nfOIawW+6yT1GtF07V/VVu7bqgw+6aqCPPnKN5FdeqbprV+r3Lcb06a6M999XLVFCde/ei/Oio1X7\n93exDBzoHixkjGri1VABP6n76hWoZLFkiWrFiqlb59Zb3bOGvX0C3b/fnZhHjnRPi5s717vlp0RE\nhDseU6b4f9vJmTxZNUcO1e++u3T6lCmqtWol/jzkiAjXu6lsWdXff09/HOfOue/m008v7Y31/vvu\n8bBpOZHPnHlp+0O/fqr167v2lX//VX3gAfeM69DQ9MdvLi+JJQurhvKy775zQ2X//XfKH9BSsiTU\nqQOlS7u+7N6g6h4AlDevGxMpJMT1AipfHj75xHvDRifnvffcfQa//OKf7aXWkSNQpMildyNHR7sn\nuH35ZfyqmwMHoGVLN4DeiBGQwIMMvSY62rVxNGrkbl5MqQ0b3CB/kya5u9ZjyrrvPihUCNauhRo1\nXHVczpy+id1kXFYN5Sf9+rnrtWHDUrb8iROqefK4q4oiRVQ3bUrZetu3J/7LV9X1vqlZ89LnG589\nq/rhh67q5MUX3Wdviopyz1f+91/3efdut62dO727HX8YP95VT8WO/cQJ1WrVVN98M+lj701797rq\nssWLU7Z8VJRqw4aqQ4bEn3f8uPub+Phj/8VvMh6sGso/HnvMdaNs2zZlyy9f7v4Dq7oT+T33JL/O\nsWOqISGqb72V8PxTp1RLlbr05rDYDh5UvfdeF2dKqyH++kt169aE50VHuyqdG29ULVNGNXdu1Tp1\nVKtXTzzGjOCzz1xby549ruqpSRPVJ5/0/4n2l19c4jp6NPllR4xwxz72jwRjUsOShZ/ceae7qihU\nKGX/YUeOVH34Yfc+IkK1XDnVWbOSXmfMGNVbblEtX1518OD483v3Vu3UKekyoqJcw3rx4slv7/x5\n1apVXbvKwIFu3Zgyfv7ZNdZWq+bq+qOjXZ/8BQtUv/xS9cyZpMsOdu+/r3rddar33efq+QN1En7h\nBdUWLS4e+4QcOaJatKi7E92YtLJk4SdVqqiuW6d6/fWqK1Ykv3zv3q7qKsaECe7Em9RJqX17l5B2\n7nRXELF77mzd6qp+9u9PWbxz56oWKOCuNhLz2WfuRrZt29zVyG23qX7+uTuJ1q7tYk7qJJbR9e2r\nevfdgU185865BukPPkh8mW7dVJ991n8xmcuTJQs/iI5WzZvX1W0/+6zqO+8kv86997rupLHLqF9f\nddSohJc/d851tYwZt2f9evdrsmRJ1fz5VbNmVR00KHVxd+qU+DoxbSnr17vP58+rvv226v33uzGE\nrO7bf3btct/1kCHubyzGvn2q//uf++EQFha4+MzlIbFkYb2hvOj4cdcD6uRJmDYN3n8fkhvMsWJF\n+PlnqFLl4rSlS6FdO3fDVO7cly4/f767A3jVqovTTpxwI5zmyeNeSd2tm5AZM9yjS5cvjz/vsccg\nVy749NPUlWl84/ffYcAAmDMH7r7bTZsxAzp0cKPd+vuOeHP5sTu4/eDPP6FjR1i/HsLD3V3KBw64\n7qsJOXvWPdw+LCz+Cb5tW9e98dVXL53+wgvuecdvvum9uCMjoVQpl6Suvfbi9FWrXDfRzZt920XU\npN6xY+4Jg9HRrqu2fT/GW4L5saqXjd273bhD4H7h164NCxdenH/27KXL//WX66+f0JVA//7w8cdu\n9M/YfvkF7rnHu3FnzeqeuzBu3MVpUVHwxBNuWAw7EQWfwoXhySfdqLD2/RhviIyO5ONlHyc635KF\nF4WGupN/jNtug1mz3BhA3bu7q4hNmy7O37zZ3fyVkPLl3VXKSy+5G+wAtm1zg/DVqOH92B96yD3h\nLWZbn37qroi6dfP+towxwee/v/2XGX/NSHS+JQsvCg29eGUBLlmMGOFG8SxZEjp3ds+ajpFUsgDX\njrBli0s0kZHw66/uqiKxkU3To25diIhwd/fu2AHvvgvDhtlzlo3JqCIiI9h5fCcbD29k5b6VLNq9\niDk75zB9+3TWHVp3ybKfr/ic2TtnM77t+ETLyyiPVc0QQkOhevWLn2vVgs8+c/X+hQq5kUkffBD6\n9nUn4S1b3HAOiSlQAGbPhjZt4IEH4OhRd6XhCyIXry5Wr4ZXXrHGUmMyqgOnDtD0m6ZEREaQJ3se\ncmXNRc6sOcmRNQfZQ7Kz4fAGbih6A683ep2ws2H0W9SPJY8soUDOJOo0E+oidTm8CEDX2Xr1VBct\nSnx+dLQbVC9m8LkaNVI2EN3Zs2500pw5Lw6l4QsbN7qutzfdZKOQGpNRHTx1UCsPrqz9FvRLdJmI\n8xH6xcovtMwnZTTvu3l18e6L48lgXWd9r3Rp16ModlVUXH36uK6uH3/sejUdOABXXJF82dHRrs2i\nUiWvhZugRx5xXTDtuQbGZDxHTx+l6TdNaVO5DX2a9El2+fNR5wk9Gcq1hS52g7Susz52/rzrAXX6\ntOtdlJitW91IpkuWwC23wL59fgvRGHOZORlxku/WfcfGIxvZcXwHaw+upXuN7vS7tR+SxgbHxJKF\ntVl4yd69UKJE0okC3NDgJUvCF18k3bhtjDGJOX7mOJ/+/imDVwzmtmtvo37p+rSs2JIKhStQvpBv\nGhstWXhJ3J5QSXnoIXj9dXj0Ud/GZIzxvYjICLJIFrKHJD10wunzpwk9GUroyVCOhB/hjvJ3UCR3\nkRRt43D4YT5c+iE7j+9k98nd/PXPX7Su1Jrl3Zf7LDnEZcnCS1KTLNq1gxdftCsLY4JBtEYTfi6c\nfDnyJTh/94ndfPvnt0zZNoXWlVrzXL3nyJnVPTVq2vZpPD71cZpd04xR941KdBt/H/+bGl/W4Mo8\nV3J1/qvJlz0fvab3osMNHXiu3nPJnvBfnfMqYWfDaFulLWUKlKFCoQoUzl04zfucFnafhZekJlmU\nLu26wtbnZUcWAAAgAElEQVSp49uYjDHJe3Pem1z5wZV0m9yN1QdWo6psObqFT5Z9QtNvmlJzWE0O\n/nuQt5u8zcr9K6k0uBLf/vktHX/qSK/pvfj87s+ZtXMWS/csTXQbn6/8nB41e7C913bmdJ7Dz+1/\nZtOTmyiYqyB1v6rLDxt+SHTdTUc2MWXrFIa1HEa7G9pRt3RdvycKsAZur3n8cXePRc+eftukMSad\n1h1aR7NvmzGn8xymbZ/GkJVDOBN5hhwhObir/F20qNiCu8rfRY6sOS6sM3/XfN6c/ya1StSib9O+\n5Mmeh+/Wfccnyz9hRfcVhGQJuWQb4efCKTOwDKseW0XZAmXjxbD6wGru+O4OVnRfwTUFr4k3/77v\n7+OWq2/h+frPe33/E2K9oXzsrrvgqaegRQu/bdIYk4TVB1bTaVInjoQfIU/2POTNnpfHaz3Okzc9\niYgQFR1FvRH1eKzWY3Sv2R1w4yOFngzlmgLXpKo3kapyy8hb6FytM4/VeuySeV+u+pLpf03n5/Y/\nJ7r+x8s+ZsKmCSzsupBsIdkuTF8SuoSHfnqIrU9tvVD15Wv2DG4fq1Ll4jMfjDGBEx0drcNWDdMi\n7xfRcevH6cFTB3XHPzt02Z5lWnVIVX108qMacT5CP176sTYZ1USjvfRQltX7V2vRD4rqsdPHLoml\nyudVdM7OOUmuGxUdpXeMvkP/N+d/l6zbYEQDHblmpFfiSynspjzfUXU31u3bl7Ib7IwxvvHvuX95\natpTrNq/iokPTqRSkUvvYj119hSdf+7MwX8Psv3YdpY9uowKhSt4bfs9p/Zk36l9jLpvFIVyFWLO\nzjk8M+MZ1vdcn+yVyqF/D1Hjyxrcfu3tiAgnIk6w/dh2/vzPn/GqtnzJqqF86J9/oFw5d2e2Mcb3\nQk+GMn37dO6rdB/F8hYDYNmeZXSa1InGZRoz6K5B5MmeJ8F1ozWadxe9S7E8xehRq4dX4zp9/jQv\nz36ZHzf/yLB7hjFs9TBaVGgRr2oqMRsOb2DZnmWEZAkha5asNLy6IeUKlvNqjMmxZOFDa9e6EWXX\nrUt+WWNM+hwOP0zDr91J9Pd9v9O4TGPKFijL9xu+Z2iLodxf+f5Ah8j8XfPpNrkbYWfDCH02NNHE\nFYzsDm4fSk23WWNMyh07fYysWbKSP2d+wFUjtRjbgnbXt6PvrX05dfYUEzZNYM2BNaz9z1qK5y0e\n4IidJmWbsO4/6/j7xN8ZKlEkxa4svGDwYNi4EYYO9cvmjLnsRURG8PGyj/lo2UdERUdxZ/k76Vyt\nMwOXD6RM/jIMazkszWMfmaTZY1V9KO4T8owxaTd5y2SqfF6FlftXsrLHSnY+s5PGZRrTd2FfiuYp\nytB7hlqiCAC7svCCDh3cE+weftgvmzPmsnQ4/DC9pvdizYE1DGkxhOblmgc6pEzJrix8yNosjEm7\nI+FHGLlmJDcOvZGy+cvy53/+tEQRhKyB2wv27LFkYUxKRGs0m49sZsHuBSwOXczyvcs5HnGcm0vd\nzJQOU6hTygZMC1ZWDZVOkZGQOzeEh0O2bMkvb0xmNffvubSf2J58OfLRuExjGpVpRN3SdalYuCJZ\nxCo5gkWG7DorIruAk0A0cF5V64hIQeAHoAywC3hQVU8GKsb9+6FoUUsUxgDsC9tH18lduabANXxx\nzxcXkkDoyVAe/ulhxrQew23X3hbgKE1aBHs6jwaaqGoNVY25Pn0ZmK2q1wFzgVcCFh2uCuqqqwIZ\ngTHB4ddtv1JrWC0aXtWQbce28eiUR4mKjiIiMoI249vwfL3nLVFkYEFdDSUifwO1VfVYrGlbgMaq\nekhEigPzVbVSAuv6pRpq3Dj4+Wf4IfHh6I25rEVFR/Hy7Jf5YeMPjGk9hlvK3EL4uXDuGXcPZfKX\nIVuWbJw4e4LxD4y3Lq8ZQIashgIUmCUiUcCXqvoVUExVDwGo6kERKRrIAK0nlMnMws6G0X5ie85F\nnWPN42suPJQnT/Y8TO0wlZbjWnLg3wOs6L7CEkUGF+zJooGqHhCRK4GZIrIVl0BiC+il0Z49ULFi\nICMwJjB2Ht9Jy3EtaVymMZ/e+eklz2EAlzBmdJxBRGREoo8sNRlHUCcLVT3g+feIiPwM1AEOiUix\nWNVQhxNbv0+fPhfeN2nShCZNmng9xtBQaG5dwk0ms//Ufm4ZeQuvNHyFp+o8lehy2UOykz0kux8j\nM6k1f/585s+fn+xyKWqzEJH6QFliJRdV/Tbt4SVPRHIDWVT1XxHJA8wE3gKaAf+o6gAR6Q0UVNWX\nE1jfL20WNWrAiBFQs6bPN2VMUDgfdZ5bv72V28rdxhuN3wh0OMbL0txmISKjgWuBtUCUZ7ICPk0W\nQDFgkogoLs4xqjpTRFYB40XkEWA38KCP40hSaKj1hjKZyytzXiFv9ry81ui1QIdi/CjZKwsR2QxU\n8dtAS17ijyuL8HAoUgROnwZruzOZwU+bf+K/v/2XPx7740Jjtrm8pGdsqA1AcAwSH2Ri7rGwRGEu\nJ7N2zKLXtF5EREZcMn3R7kU8PvVxxrcdb4kiE0pJsigCbBKR30RkSszL14FlBNZt1lxuDocfpsvP\nXdh4ZCPNv23OkfAjAEzcNJE249swtvVYG78pk0pJb6g+vg4io7IBBM3lRFXp8UsPOt3Yif7N+/P6\n3NepO6IuD93wECPXjmRmp5lUL1490GGaAEkyWYhICNBHVZv6KZ4MxRq3TUa27tA6coTk4Loi1wEw\nYs0IQk+GMv6B8WSRLLzT7B0qFq7I8NXDWfzIYsoWKBvYgE1AJZksVDVKRKJFJH8gB+sLVqGh0LBh\noKMwJnXOnD/Da3NfY+yGsQCUyFuC+yvdz6AVg5jfZT45sua4sGyX6l3oUr1LoEI1QSQl1VD/AutF\nZBYQHjNRVZ/2WVQZhFVDmYxm+d7ldP25K9WLV2d9z/UUzFmQBbsX8P2G7/ngtg+4vuj1gQ7RBKmU\ndJ1N8GeFqn7jk4i8xB9dZytWhClToFK8YQyNCS57w/by6pxXmbVzFoPuHETb69sGOiQTpBLrOhvU\no86mh6+Thap76NHRo5Anj882Y0y6RGs0fRf0ZdCKQfSs3ZPeDXrbOE0mSem5g/tvEhisT1XLeSm2\nDCkmSViiMMFKVek1rRfrDq9j7eNruSq/9cYwaZeSNovasd7nBNoChXwTTsZhPaFMsHt1zqus2L+C\nOZ3ncEWOKwIdjsngkr0pT1WPxXrtU9WBQAs/xBbUrHHbBLP+i/rzy7ZfmPHwDEsUxitSUg0VezzV\nLLgrjaAe2twf7MrCBJujp48ydv1YRq4dydnIs8zuPNuG5TBek5KT/kex3kcCfxPgkV6DgT172wST\niZsm0n1Kd1pe15IPb/uQptc0JYukZDQfY1ImJcniUVXdGXuCiFzjo3gyjL17obqNfGCCwJ6Te3ji\n1yeY3Xk2tUvWTn4FY9IgJT89JqZwWqaybx+UKhXoKExmF63RdJ3clWdufsYShfGpRK8sRKQScD2Q\nX0Rax5p1Ba5XVKZmycIEg4HLB3I28iwvN4z3sEhjvCqpaqjrgHuAAkDLWNNPAT18GVSwU4X9+y1Z\nGN84GXGSoauGclf5u6hWvFqCy4SfC2fGXzPov7g/K7qvICRLiJ+jNJlNSob7qKeqy/wUj9f48g7u\nY8egfHk4ftwnxZtMbNmeZTz000PULFGTpXuW0uCqBrzR+A1yZ8vNnwf/ZO3BtSwMXcgf+/+gVsla\nvFT/JVpUzPQ92Y0XpfkObuCYiMwBiqnqDSJyI3CvqvbzepQZxN69ULp0oKMwl5Oo6Cj6L+7P4BWD\n+fKeL2lVqRXh58IZumoot42+jRwhOahevDrVilWjd4PeNCrTiLzZ8wY6bJOJpOTKYgHwIvClqtbw\nTNugqjf4Ib408+WVxbRpMGgQzJjhk+JNBvX38b/Jkz0PRfMUTdV6u0/spuOkjmTLko3R94+m1BWX\n1m+qKmLP7jV+kp5ncOdW1RVxpkV6J6yMyRq3TVxfr/maWsNqUWlwJa784EqaftOUDYc3JLmOqjJ2\n/VhuGn4T91a8l9mdZ8dLFIAlChMUUlINdVRErsUzmKCIPAAc8GlUQc6ShYmhqgxYMoAv//iS5d2X\nU6FQBQ6FH+LrNV/T89eeLOy6MN7JfveJ3YxdP5Yx68egKL91/I0aJWoEaA+MSZmUXFk8CXwJVBKR\nfcCzQE+fRhXkLFkYcIni+ZnPM2b9GJY8soSKhSsiIhTPW5zeDXrz77l/Gb9x/CXrvL3gbWoNq8Xu\nk7sZ2mIo63uut0RhMoRkryw8d283F5E8QBZVPeX7sILbvn3QqlWgozCBNmDJABbuXsjCrgspmKvg\nJfNCsoTw6Z2f0mlSJ1pe15Lc2XLz7Z/fMmrtKDY9uSnV7RrGBFqSVxYiEiIiRQBUNRw4KyI9RGSz\nX6ILUnv32pVFZjd5y2QGrxjM5PaT4yWKGI3KNKJu6bp8sOQDFu5eyAszX2DqQ1MtUZgMKdHeUCLS\nHlf9FA5sB94BvgZWAn1VdbW/gkwLX/aGKlwYtmyBK6/0SfEmyK07tI5m3zbj14d+pU6pOkkuu/vE\nbmoOq3mhp9Nt197mpyiNSZtUP1ZVRDYA96nqX55hypcBD6jqL74N1Tt8lSzOnIGCBd2/1kkl8Pac\n3EOURlG2QFm/bO/o6aPcNPwm3rn1HR6q+lCK1vli1RfkzZ6Xjjd29HF0xqRfWpLFalWtGetz0N9b\nEZuvksWOHdC8Ofz9t9eLNqk0afMkHpv6GIJQIl8J7q14Lx2qdqDKlVV8sr2o6CjuGnMXNYrXYMBt\nA3yyDWMCLS13cBcVkf/G+lwg9mdV/dibAWYU1hMq8KI1mn4L+zF89XCmPTSNmiVqsnzvciZvnUyz\nb5tRrVg1nq/3PM3LNffqPQpvLXiLyOhI3mn2jtfKNCajSOrK4s2kVlTVt3wSkZf46spi3Dj4+Wf4\n4QevF20SEREZwdsL3mZv2F7ORJ4h9GQoWSQLPz34EyXylYi37Nj1Y/lo2UfkzJqTAc0H0Lxc83TH\n8Ou2X/nPr/9hVY9VFMtbLN3lGROsUl0NldH5Kll88AEcOAAfZ8rrKv87ff40931/H3mz56XVda3I\nlS0XebPnpdk1zciRNUei66kqEzdN5NW5r1KuYDk+uO0Dbix2Y6q3Hxkdydy/59JpUid+evAnGlzd\nID27Y0zQS89AgiaWffvscar+En4unJbjWlIyX0lG3TeKrFlS/ucqIrS9vi2tKrVi+B/DafZtMxZ0\nXZBke8bqA6tZumcpIRJCFsnCmoNr+GnzT1yd/2o+u+szSxQmU7NkkUr79kG9eoGO4vJ3OPwwbSe0\npVzBcnzV8qs0P68he0h2nqzzJLmy5aLthLas6L6CPNnzxFtOVXn4p4e5qeRN5MmWhyiNonyh8izv\nvpxyBculd3eMyfAsWaSSNXD7lqoyfuN4npnxDN2qd+OdZu+QRVIyKk3SulXvxvxd8+k1vRdft/o6\n3vxNRzYRfi6cb+77xgbuMyYBKf5fKCJ1RWSGiMwXkft8GVQws2ThO2Fnw3hgwgO8teAtJrefTP/m\n/b2SKMBVSw1pMYRle5fxzdpv4s2fsGkCD1R5wBKFMYlI9H+iiBSPM+m/wP3A3UBfXwYVrKKjXeN2\nyZKBjuTy9PaCtwmREFY/vpqbS9/s9fLzZs/LhLYTeH7m8+w/tf+SeRM3TeSBKg94fZvGXC6S+tn2\nhYi8ISI5PZ9PAA/gEkaYzyNLgojcKSJbRGSbiPT213YPH4b8+SFH4p1wTBrtPrGbkWtH8umdn5Iz\na87kV0ijG4reQJdqXXh30bsXpm0+spkTESeoW7quz7ZrTEaXaLJQ1fuANcBUEemMG5o8B1AYCFg1\nlIhkAQYDdwDXAx1EpFJK1j1/Hs6dS/u2rQrKd16f9zpP3vRkvPsmfOHlhi8zbsM4dp/YDbirijaV\n23itysuYy1GS/zs840DdAeQHJgHbVHWQqh7xR3CJqANsV9Xdqnoe+B5IcMDw06cvvo+MhBYtoE4d\nOHgw8cI3bID774c+fWDFClf1FMOShW/8efBPZu6YyYv1X/TL9q7McyU9a/ek70JXmzph0wTaXt/W\nL9s2JqNKqs3iXhGZB8wANgDtgFYi8r3nyXmBUgrYE+vzXs+0eHrHqqB64QU38F/r1tCgAfz1V/zl\nx4+Hpk2hYUMID4du3Vz7xKRJbv6+fVC6tNf2w3j0nt2b1xq9Rr4c+fy2zefrPc/krZOZum0qR08f\npf5V9f22bWMyoqS6zvbD/YrPBfymqnWA50WkAm648vZ+iC9dJk+Ge+5xz5+YNg1+/92NGFuiBDRq\nBJ995togAKZPh59+gpkzoYbnwWUffODWadsW1q511Vh2ZZF+c/+ey8wdMzkbeZbjEcf565+/eKzW\nY36NoWCugjx787N0+LEDXat1tSooY5KRVLI4CbQGcgOHYyaq6nYCmyj2AVfH+lzaMy2epk370KYN\nqMLQoU0oWLAJAD16QNGiMGiQWy4q6ymuuGoXq1ZVpXDhS8u4+WZYuRIeeMD9+/nnXt+fTGX2ztk8\n9ONDPH3z0xTNU5RrC13Li/VfJHtIdr/H8vTNTzN45WDa3dDO79s2JljMnz+f+fPnJ7tcUgMJFgE6\nAOeBsaoa0B5QMUQkBNgKNAMOACuADqq6Oc5yqqoMHAg33OCGFY9rb9heBv0+iBFrRhAVHUXoc6Fc\nkeOKBLd77hy8+y48/DBUqODtvQpuK/at4KNlH3HntXfSrUa3NJezfO9y7h13LxMfnEijMo28GGHa\nRURG+LT3lTEZzWU1kKCI3Al8imtzGaGq7yWwTJIDCY5ZN4Ze03vRuVpnnrn5GV6a/RJNyzbliZue\n8F3gGczSPUt5dc6r7Dqxi/Y3tGfs+rHseHoH2UKypbqs9YfW03x0c0a2GsndFe72QbTGGG+4rJJF\nSiSVLLYc3ULDrxsyr8s8qharCsC8v+fRa3ov1vdcb3fxAsdOH6PKkCq83/x9Hqr6ENlCstH0m6b0\nqNkjxU+Ii6GqVPvCPWOiS/UuPorYGOMNiSWLy7pVb9HuRfGmnTl/hnYT2/Fus3cvJAqAJmWbEKVR\nLA5d7M8Qg9Zrc1+jbZW2dKne5cKVxPP1nuejZR+R2h8YS/cs5WzUWTpX6+yLUI0xfnBZJ4sHJz7I\nq3Ne5UTEiQvTnp/5PJWKVKJHzR6XLCsi9KzdkyGrhvg7zKDzx/4/mLRlEn2bXjqqy90V7ib8XDgL\ndi9IVXlf/PEFj9d63K7YjMnALutqqIOnDvLU9KeYvn06BXMVpHyh8oSeDGX1Y6vJnzN/vHVORJzg\nmk+vYcuTWzLt09CiNZoGXzege43uPFrz0Xjzv1z1JVO3T+WXDr/Em6eqrNi34pJxnY6ePkr5QeXZ\n8fQOCucuHG8dY0xwyZTVUMXyFmNC2wmEvRLGgq4LeK7uc8zpPCfBRAFQIGcB2lZpy4g1I+LN+/fc\nvwxZOYS9YXt9HbbP7Qvbx0+bf0pw3jdrvyFaoxPt9dS5WmdW7FvBlqNb4s37dfuv1B1Rl1FrR11S\nXqtKrSxRGJPBZYrnWWSRLJQrWC5FD7HpWbsnd425C0GoWaIm5QuV57t13zF45WAK5izI9mPb+eTO\nT/wQtfeFnQ3j/SXvM3TVUPJky8OGwxt4o/EbF+Yv2LWA3rN7M+3haYnepJYrWy561u7JO4veYfT9\noy9Mj9Zo/jf3f/Rv1p8XZ71I1aJVqVGiBl/+8SXf3Bd/SHBjTMZyWV9ZpEWNEjUY3nI4x84c470l\n71H/6/qEngxlySNL+K3jb3y3/jsiIiMCHWaqrdi3goqfVWRP2B7WPL6GlT1WMm7DOPov6g/Aj5t+\npO2EtoxrM47aJWsnWdYL9V9g2Z5lTNo86cK0CRsnkDNrTno36M0XLb6gzfg2jN84nlzZctlorsZc\nDlT1sny5XfO+20ffrmPWjfFJ2b4SGRWp1b+ort+u/faS6fvD9muFQRW0zQ9ttORHJXX1/tUpLnNp\n6FIt9kEx3R+2X89HndcKgyrorB2zLsx/edbLmuWtLDpkxRCv7Ycxxvc8585451S7skilHjV7MHz1\n8ECHkSoj1owgb/a8dLyx4yXTS+Qrwbwu88gekp1F3RZRo0SNFJdZ76p6PF7rcbpN7saotaMofUVp\nml3T7ML8frf2o2/TvvG2aYzJmC7r3lC+2LdzUee46pOrWNRtERULV/R6+d52/MxxKn9emekPT09V\nMkiJ81HnaTiyIX8e/JN5XeZR76p6Xi3fGON/mbI3lC9kD8lO12pd+Wr1V4EOJZ7zUed57JfH6DGl\nBzuP7wTgrQVvce9193o9UQBkC8nGmNZjeKPxG5YojLnM2ZVFGmw/tp2GIxuy57k9ARktNSHno87T\n4ccOnI06S43iNRiycgi3X3s7s3bOYtMTm7gyz5WBDtEYkwHYlYUXVShcgSpXVmHylsmBDgWAyOhI\nHv7pYc5EnmFi24m83fRttvfaTqUilfjsrs8sURhj0s2uLNJo6MqhrNy/kq9bfe2zbaREVHQUHSd1\n5ETECSa1m2TDbRtj0sWuLLysUZlGLNy9MKAxqCqPT32cw+GHLVEYY3zKkkUaVb6yMiciTrAvLMGH\n9PmcqvLf3/7LpiObmNx+siUKY4xPWbJIoyyShVvK3MKi0PjDoPvD2wveZv7u+Ux7eBp5s+cNSAzG\nmMzDkkU6NLq6EQt2pW64bm/YcHgDQ1cN5beOv1EgZwG/b98Yk/lYskiHRmUasTDU/+0W/Rf359m6\nz1I0T1G/b9sYkzlZskiH6sWrsy9sH0fCj/htm3/98xe//fWbPSvcGONXlizSISRLCPWvqu/XR7EO\nWDyAJ256gityXOG3bRpjjCWLdPJnF9q9YXv5cfOPPHPzM37ZnjHGxLBkkU6NyjRK9TOp0+rDpR/y\nSI1H7Klzxhi/yxRPyvOl2iVrs+3YNk5GnEz0ca1ptf3Ydl6f9zrh58M5G3mWlftXsvGJjV7dhjHG\npIQli3TKHpKdOqXqsGTPEu6ucLfXylVV/vPrf6herDqNyzYmZ9aclMlfhpL5SnptG8YYk1KWLLyg\nUZlGzP17rleTxcRNEzkSfoQBtw0gaxb7mowxgWVtFl7Q8caOjFo76sIzJNIr/Fw4L8x6gc/u+swS\nhTEmKFiy8ILyhcrTu0Fvuk/pjjdGun1v8Xs0uKoBjcs29kJ0xhiTfpYsvOS5es/x77l/L3k+98mI\nk0zfPj1V5ez4ZwdDVw3lg9s+8HaIxhiTZvY8Cy/acHgDTb9pytJHljJl6xTeX/o+/577l+WPLqdq\nsaopKqPXtF4UyFmAvrf29XG0xhgTnz3Pwg9uKHoDT9d5msqfV2ZR6CLmdJ7DKw1fYeDygSla/3zU\neX7Y+APdanTzcaTGGJM6dmXhZZHRkfz1z19UKlIJgKOnj1LhswpseXILxfIWS3LdX7f9yruL32XJ\nI0v8EaoxxsRjVxZ+kjVL1guJAqBI7iI8WOVBhq4amuy6363/jo5VO/oyPGOMSRO7svCDzUc20+Sb\nJux+dneiT7QLOxvGVZ9cxc6nd9pwHsaYgLEriwCqfGVlapWoxZh1YxJdZtLmSTQp28QShTEmKFmy\n8JPn6j7HJ8s/SfQ+jNHrRlsVlDEmaFmy8JPm5ZoTGR3J7/t+jzdvX9g+/jjwB/dUvCcAkRljTPKC\nMlmIyJsisldEVnted8aa94qIbBeRzSJyeyDjTA0RoW2Vtvy46cd488ZtGEfrSq3JlS1XACIzxpjk\nBWWy8PhYVWt6XjMARKQy8CBQGbgLGCIi8RpiglWbKm34cfOPl1RFqSoj1oygS/UuAYzMGGOSFszJ\nIqEk0Ar4XlUjVXUXsB2o49eo0qFasWqICGsPrr0wbcmeJagqt1x9SwAjM8aYpAVzsnhKRNaKyFci\nEvNUoVLAnljL7PNMyxBEhDaV3dVFjOGrh9O9Zncy0AWSMSYTCtj41yIyC4h9S7MACvwPGAK8raoq\nIv2Aj4Duqd1Gnz59Lrxv0qQJTZo0SUfE3tGmchu6Tu5Kv1v7cSLiBJO3TObD2z4MdFjGmExq/vz5\nzJ8/P9nlgv6mPBEpA/yiqjeKyMuAquoAz7wZwJuqGq+LUTDdlBdbtEZTZmAZfuv4G/P+nseC3QsY\n33Z8oMMyxhggg92UJyLFY31sDWzwvJ8CtBeR7CJyDVAeWOHv+NIji2ShdaXW/LjpR4avHk6Pmj0C\nHZIxxiQrWB/D9r6IVAeigV3A4wCquklExgObgPPAE0F5+ZCMNlXa0PqH1lyR4wqalWsW6HCMMSZZ\nQZksVLVzEvP6A/39GI7XNbiqASFZQni0xqNkkaC8uDPGmEsEfZtFWgVrm0WMJaFLqFqsKlfkuCLQ\noRhjzAWJtVlYsjDGGHNBhmrgNsYYE1wsWRhjjEmWJQtjjDHJsmRhjDEmWZYsjDHGJMuShTHGmGRZ\nsjDGGJMsSxbGGGOSZcnCGGNMsixZGGOMSZYlC2OMMcmyZGGMMSZZliyMMcYky5KFMcaYZFmyMMYY\nkyxLFsYYY5JlycIYY0yyLFkYY4xJliULY4wxybJkYYwxJlmWLIwxxiTLkoUxxphkWbIwxhiTLEsW\nxhhjkmXJwhhjTLIsWRhjjEmWJQtjjDHJsmRhjDEmWZYsjDHGJMuShTHGmGRZsjDGGJMsSxbGGGOS\nZcnCGGNMsgKWLETkARHZICJRIlIzzrxXRGS7iGwWkdtjTa8pIutEZJuIDPR/1MYYkzkF8spiPXA/\nsCD2RBGpDDwIVAbuAoaIiHhmDwUeVdWKQEURucOP8XrF/PnzAx1CgoIxrmCMCSyu1LK4UidY4wpY\nslDVraq6HZA4s1oB36tqpKruArYDdUSkOJBPVVd6lvsWuM9vAXtJsP4hBGNcwRgTWFypZXGlTrDG\nFYxtFqWAPbE+7/NMKwXsjTV9r2damqX0S0nJcikta9euXV4rKxjj8mbsKYkppWVZXKkry+Ly/vYy\n+pXCogUAAAcfSURBVDnCp8lCRGZ52hhiXus9/7b05XZTyv4QLrJkkfLlLK7ULZeR48rI/xdTulxK\nyxJVTdGCviIi84DnVXW15/PLgKrqAM/nGcCbwG5gnqpW9kxvDzRW1Z6JlBvYHTPGmAxKVeM2D5A1\nEIEkIHZgU4AxIvIJrpqpPLBCVVVETopIHWAl0BkYlFiBCe2sMcaYtAlk19n7RGQPUBeYKiLTAVR1\nEzAe2ARMA57Qi5c/TwIjgG3AdlWd4f/IjTEm8wl4NZQxxpjgF4y9oRIkIqVFZK6IbPQ0lD/tmV5Q\nRGaKyFYR+U1E8sdaJ7Gb+zp4GtrXisg0ESkUJHG1E5E/PeX0T2tMaYlLRAp5lj8lIoPilOWVmyG9\nHFM/EQkVkbC0xuPtuEQkl4hM9Xyv60Xk3WCIyzNvuois8ZQT+96lgMYVq8wpIrIurTF5Oy4RmSci\nWzzHbLWIFAmSuLKJyJeedTaJyP1pjSvVVDVDvIDiQHXP+7zAVqASMAB4yTO9N/Ce530VYA2uXaYs\n8BeubSQEOAQU9Cw3AHgjCOIqhGvEL+RZbiTQ1I9x5QbqA48Bg+KU9Ttwk+f9NOCOIIipDlAMCAvA\n31aCcQG5cJ0u8Hy/C9N6rHxwvPLGej8ReDAY4vLMvx/4DlgXDN+jZ948oEZ6/7Z8EFcf4O1Ynwt5\nI8YU7Ye/NuT1wOFnoDmwBSgW60vZ4nn/MtA71vLTgZs9/4kPAVfjTtJDge5BEFdtYFas6R2Bwf6K\nK9ZyXbj0BFgc2BTrc3tgaCBjijMv3cnCF3F55g/EjTgQNHEB2XCdSNoGQ1xAHlxSrUQ6k4WX45oH\n1PL235YX4goFcvkiruReGaYaKjYRKQtUB5bjDvYhAFU9CBT1LJbgzX2qGgk8gRtuZC9uWJERgY4L\nd4VxnYhcLSJZcXenX+XHuBLj9ZshvRCTz3grLhEpALQE5gRLXOK6oR8EwnBXF8EQV1/gQ+CMN+Lx\nYlwAozxVUK8FQ1xysSq7n4j8ISI/iMiV3ootORkuWYhIXtwf+jOq+i8Qt4U+yRZ7z4m4J1BNVUvh\nksargY5LVU944hqPGy/rbyAq0HH5QjDGBN6LS0RCgLHAQHVD1gRFXKp6J1ACyAHcGui4RKQacK2q\nTsFd5Xulu7uXjtdDqloVuAW4RUQ6BkFcWYHSwGJVrYVLOB+lN66UylDJwnOinwiMVtXJnsmHRKSY\nZ35x4LBn+j4u/WVe2jOtOu6mv12e6eOBekEQF6r6q6rWVdUGuO7B2/wYV2ISjTeAMXmdl+MaBmxV\n1c+CLC5U9RyuGqpVEMRVD6glIjuBRbjBQecGQVyo6gHPv+G4xF8n0HGp6jEgXFUneSZNAGqkJ67U\nyFDJAvgaV3/+aaxpU4CunvddgMmxprcXkewicg2em/twJ7oqIlLYs9xtwOYgiIuYS0oRKYirKvvK\nj3HFduEXnufy+KSI1PH0oOmcyDp+iymF0wMSl4j0A65Q1eeCJS4RyeM5GcWctFrg6ssDGpeqfqGq\npVW1HNAQl2DTe8XjjeMVEnN+EJFswD3AhkDH5fGLiDT1vG+Oux/NPwLRUJKWF9AAVy2zFtebaDVw\nJ64X0WxcD4OZQIFY67yCawvYDNwea/pjnoO8FvcFFQySuMYCG3F/mOlqgExjXH8DR3F12qFAJc/0\nWrjquu3Ap0ES0wBc20+kZ3p6erR5JS5cW0605zuMKeeRIIirKO4HyVpgHfApkCXQccUpswzp7w3l\nreOVG1jlKWc98Amee9ICfbxwHXMWeMqaBf9v7/5Zo4iiMIw/r0jQwlilE0Qx2KiIIFgIdoKNlRZa\npEsjiqAfwE7BRgSxFhsre8VOsBAkQvALWCkqFkoa/+RYzI2ESHIj2WRRnh8M7M7eXe5s8zJ3Zs5h\nz0b+s7/ZfChPktT1ry1DSZLGwLCQJHUZFpKkLsNCktRlWEiSugwLSVKXYSGNQJKfrY7Qm1bW+lp7\niHGt7+xNcmGr5ihthGEhjcZCVR2rqkMMVQHOMPSOX8s+4OKmz0waAcNCGrGq+sRQJeAy/D6DeJ7k\nVdtOtKG3gJPtjORqkm1Jbid5maEx1+y4jkFaySe4pRFI8qWqJlfs+wwcBL4Ci1X1LckB4FFVHU9y\nCrheVWfb+FlgqqpuJpkAXgDnqurt1h6N9Kft456A9B9bumYxAdxLcpShRtD0KuNPA4eTnG/vJ9tY\nw0JjZ1hImyDJfuBHVX1McgN4X1VHWq+L1Rr9BLhSVc+2bKLSOnnNQhqN5SWupxja9S71s9gNvGuv\nZxj6wMOwPLVr2W88BS61MuIkmU6yczMnLa2XZxbSaOxIMsew5PQdeFhVd9pn94HHSWaAJ8BC2z8P\nLCZ5DTyoqrut7eZcu+32A0N7XWnsvMAtSepyGUqS1GVYSJK6DAtJUpdhIUnqMiwkSV2GhSSpy7CQ\nJHUZFpKkrl+CSWM1ab4okwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x161ff8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "test_data[\"p_up\"] = pd.Series(predicted_label_prob[:,0,0], index = test_data.index)\n",
    "test_data[\"p_down\"] = predicted_label_prob[:,0,1]\n",
    "test_data['long_entries'] = np.where((test_data.p_up > 0.50) , 1, 0)\n",
    "test_data['short_entries'] = np.where((test_data.p_down > 0.50) , -1, 0)\n",
    "test_data['positions'] = test_data['long_entries'].fillna(0) + test_data['short_entries'].fillna(0)\n",
    "test_data[\"pnl\"] = test_data[\"Adj Close\"].diff().shift(-1).fillna(0)*test_data[\"positions\"]/np.where(test_data[\"Adj Close\"]!=0,test_data[\"Adj Close\"],1)\n",
    "monthly = test_data.pnl.resample(\"M\").sum()\n",
    "avg_return = np.mean(monthly)\n",
    "std_return = np.std(monthly)\n",
    "sharpe = np.sqrt(12) * avg_return / std_return\n",
    "drawdown = create_drawdowns(monthly.cumsum())\n",
    "\n",
    "print(\"TRADING STATS\")\n",
    "print(\"AVG Monthly Return :: \" + \"{0:.2f}\".format(round(avg_return*100,2))+ \"%\")\n",
    "print(\"STD Monthly        :: \" + \"{0:.2f}\".format(round(std_return*100,2))+ \"%\")\n",
    "print(\"SHARPE             :: \" +\"{0:.2f}\".format(round(sharpe,2)))\n",
    "print(\"MAX DRAWDOWN       :: \" + \"{0:.2f}\".format(round(drawdown[0]*100,2)) + \"%, \" + str(drawdown[1]) + \" months\" )\n",
    "print(\"Correlation to SPY :: \" + \"{0:.2f}\".format(round(np.corrcoef(test_data[\"pnl\"], test_data[\"diff\"])[0][1],2)))\n",
    "(monthly.cumsum()*100).plot()\n",
    "(monthly_spy.cumsum()*100).plot()\n",
    "plt.legend([\"NN\", \"SPY\"],loc=2)\n",
    "plt.ylabel(\"% Return\")\n",
    "plt.title(\"TRADING SPY OUT OF SAMPLE\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The plot above shows the % returns when we trade every day using SPY and NN based models as compared to a confidence based trading show in previous plot. With frequent trading the volatility is higher and transaction fees (not accounted in this plot) will greatly eat into any profits.\n",
    "\n",
    "\n",
    "## Suggested exercises\n",
    "Try changing the batch size, the network itself, the activation functions, and many other features and see how much it affects the output. Notice how it can change dramatically? This means what you have been training on might be noise and you need to refine either the model or more likely the inputs to the network itself. \n",
    "\n",
    "After you have completed the aforementioned exercise, experiment with applying the trained network to other to other timeseries data. Pick a stock in the S&P 500, pick something that is uncorrelated with the S&P 500, try a completely random dataset. How do you expect each of those to do? Should the performance match up with S&P 500 out of sample performance? How about the random data?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.51\n"
     ]
    }
   ],
   "source": [
    "# Repeatable factor (used for runtime verification)\n",
    "print(error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Appendix\n",
    "A great resource for getting started as a Quant: https://www.quantstart.com/\n",
    "An online back-testing and open source trading platform, hosts quant competitions and a forum: https://www.quantopian.com/\n",
    "[A summary of backtesting and biases](https://www.quantstart.com/articles/successful-backtesting-of-algorithmic-trading-strategies-part-i)\n",
    "[For those who want to automatically trade in python](https://github.com/blampe/IbPy)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}