{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from IPython.display import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CNTK 203: Reinforcement Learning Basics\n",
    "\n",
    "\n",
    "Reinforcement learning (RL) is an area of machine learning inspired by behaviorist psychology, concerned with how [software agents](https://en.wikipedia.org/wiki/Software_agent) ought to take [actions](https://en.wikipedia.org/wiki/Action_selection) in an environment so as to maximize some notion of cumulative reward. In machine learning, the environment is typically formulated as a [Markov decision process](https://en.wikipedia.org/wiki/Markov_decision_process) (MDP) as many reinforcement learning algorithms for this context utilize [dynamic programming](https://en.wikipedia.org/wiki/Dynamic_programming) techniques.\n",
    "\n",
    "In some machine learning settings, we do not have immediate access to labels, so we cannot rely on supervised learning techniques. If, however, there is something we can interact with and thereby get some feedback that tells us occasionally, whether our previous behavior was good or not, we can use RL to learn how to improve our behavior.\n",
    "\n",
    "Unlike in supervised learning, in RL, labeled correct input/output pairs are never presented and sub-optimal actions are never explicitly corrected. This mimics many of the online learning paradigms which involves finding a balance between exploration (of conditions or actions never learnt before) and exploitation (of already learnt conditions or actions from previous encounters). Multi-arm bandit problems is one of the category of RL algorithms where exploration vs. exploitation trade-off have been thoroughly studied. See figure below for [reference](http://www.simongrant.org/pubs/thesis/3/2.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://cntk.ai/jup/polecart.gif\" width=\"300\" height=\"300\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Figure 1\n",
    "Image(url=\"https://cntk.ai/jup/polecart.gif\", width=300, height=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem**\n",
    "\n",
    "We will use the [CartPole](https://gym.openai.com/envs/CartPole-v0) environment from OpenAI's [gym](https://github.com/openai/gym) simulator to teach a cart to balance a pole. As described in the link above, in the CartPole example, a pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center. See figure below for reference.\n",
    "\n",
    "**Goal**\n",
    "\n",
    "Our goal is to prevent the pole from falling over as the cart moves with the pole in upright position (perpendicular to the cart) as the starting state. More specifically if the pole is less than 15 degrees from vertical while the cart is within 2.4 units of the center we will collect reward. In this tutorial, we will train till we learn a set of actions (policies) that lead to an average reward of 200 or more over last 50 batches.\n",
    "\n",
    "In, RL terminology, the goal is to find _policies_ $a$, that maximize the _reward_ $r$ (feedback) through interaction with some environment (in this case the pole being balanced on the cart). So given a series of experiences $$s \\xrightarrow{a} r, s'$$ we then can learn how to choose action $a$ in a given state $s$ to maximize the accumulated reward $r$ over time:\n",
    "\n",
    "$$\n",
    "Q(s,a) = r_0 + \\gamma r_1 + \\gamma^2 r_2 + \\ldots = r_0 + \\gamma \\max_a Q^*(s',a)\n",
    "$$\n",
    "\n",
    "where $\\gamma \\in [0,1)$ is the discount factor that controls how much we should value reward that is further away. This is called the [*Bellmann*-equation](https://en.wikipedia.org/wiki/Bellman_equation).\n",
    "\n",
    "In this tutorial we will show how to model the state space, how to use the received reward to figure out which action yields the highest future reward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We present two different popular approaches here:\n",
    "\n",
    "**Deep Q-Networks**: DQNs have become famous in 2015 when they were successfully used to train how to play Atari just form raw pixels. We train neural network to learn the $Q(s,a)$ values (thus $Q$-Network ). From these $Q$ functions values we choose the best action.\n",
    "\n",
    "**Policy gradient**: This method directly estimates the policy (set of actions) in the network. The outcome is a learning of an ordered set of actions which leads to maximize reward by probabilistically choosing a subset of actions. In this tutorial, we learn the actions using a gradient descent approach to learn the policies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we focus how to implement RL in CNTK. We choose a straight forward shallow network. One can extend the approaches by replacing our shallow model with deeper networks that are introduced in other CNTK tutorials.\n",
    "\n",
    "Additionally, this tutorial is in its early stages and will be evolving in future updates.\n",
    "\n",
    "## Prerequisites\n",
    "Please run the following cell from the menu above or select the cell below and hit `Shift + Enter` to ensure the environment is ready. Verify that the following imports work in your notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "from __future__ import division\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import style\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "\n",
    "style.use('ggplot')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the following construct to install the OpenAI gym package if it is not installed. For users new to Jupyter environment, this construct can be used to install any python package. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "try:\n",
    "    import gym\n",
    "except:\n",
    "    !pip install gym\n",
    "    import gym"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Select the notebook run mode**\n",
    "\n",
    "There are two run modes:\n",
    "- *Fast mode*: `isFast` is set to `True`. This is the default mode for the notebooks, which means we train for fewer iterations or train / test on limited data. This ensures functional correctness of the notebook though the models produced are far from what a completed training would produce.\n",
    "\n",
    "- *Slow mode*: We recommend the user to set this flag to `False` once the user has gained familiarity with the notebook content and wants to gain insight from running the notebooks for a longer period with different parameters for training. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "isFast = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## CartPole: Data and Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use the [CartPole](https://gym.openai.com/envs/CartPole-v0) environment from OpenAI's [gym](https://github.com/openai/gym) simulator to teach a cart to balance a pole. Please follow the links to get more details.\n",
    "\n",
    "In every time step, the agent\n",
    " * gets an observation $(x, \\dot{x}, \\theta, \\dot{\\theta})$, corresponding to *cart position*, *cart velocity*, *pole angle with the vertical*, *pole angular velocity*,\n",
    " * performs an action `LEFT` or `RIGHT`, and\n",
    " * receives\n",
    "  * a reward of +1 for having survived another time step, and\n",
    "  * a new state $(x', \\dot{x}', \\theta', \\dot{\\theta}')$\n",
    "\n",
    "The episode ends, if\n",
    " * the pole is more than 15 degrees from vertical and/or\n",
    " * the cart is moving more than 2.4 units from center.\n",
    "\n",
    "The task is considered done, if\n",
    " * the agent achieved and averaged reward of 200 over the last 50 episodes (if you manage to get a reward of 200 averaged over the last 100 episode you can consider submitting it to OpenAI).\n",
    "\n",
    "In fast mode these targets are relaxed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: DQN\n",
    "\n",
    "After a transition $(s,a,r,s')$, we are trying to move our value function $Q(s,a)$ closer to our target $r+\\gamma \\max_{a'}Q(s',a')$, where $\\gamma$ is a discount factor for future rewards and ranges in value between 0 and 1.\n",
    "\n",
    "DQNs\n",
    " * learn the _Q-function_ that maps observation (state, action) to a `score`\n",
    " * use memory replay (previously recorded $Q$ values corresponding to different $(s,a)$ to decorrelate experiences (sequence state transitions)\n",
    " * use a second network to stabilize learning (*not* part of this tutorial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model: DQN\n",
    "\n",
    "$$\n",
    "l_1 = relu( x W_1 + b_1) \\\\\n",
    "Q(s,a) = l_1 W_2 + b_2 \\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will start with a slightly modified version for Keras, https://github.com/jaara/AI-blog/blob/master/CartPole-basic.py, published by Jaromír Janisch in his [AI blog](https://jaromiru.com/2016/09/27/lets-make-a-dqn-theory/), and will then incrementally convert it to use CNTK.\n",
    "\n",
    "We use a simple two-layer densely connected network, for simpler illustrations. More advance networks can be substituted.\n",
    "\n",
    "**CNTK** concepts: The commented out code is meant to be an illustration of the similarity of concepts between CNTK API/abstractions against Keras. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy\n",
    "import math \n",
    "import os \n",
    "import random\n",
    "\n",
    "import cntk as C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the block below, we check if we are running this notebook in the CNTK internal test machines by looking for environment variables defined there. We then select the right target device (GPU vs CPU) to test this notebook. In other cases, we use CNTK's default policy to use the best available device (GPU, if available, else CPU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Select the right target device when this notebook is being tested:\n",
    "if 'TEST_DEVICE' in os.environ:\n",
    "    if os.environ['TEST_DEVICE'] == 'cpu':\n",
    "        C.device.try_set_default_device(C.device.cpu())\n",
    "    else:\n",
    "        C.device.try_set_default_device(C.device.gpu(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "STATE_COUNT = 4 (corresponding to $(x, \\dot{x}, \\theta, \\dot{\\theta})$),\n",
    "\n",
    "ACTION_COUNT = 2 (corresponding to `LEFT` or `RIGHT`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-05-16 20:19:32,618] Making new env: CartPole-v0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(4, 2)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env = gym.make('CartPole-v0')\n",
    "\n",
    "STATE_COUNT  = env.observation_space.shape[0]\n",
    "ACTION_COUNT = env.action_space.n\n",
    "\n",
    "STATE_COUNT, ACTION_COUNT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: in the cell below we highlight how one would do it in Keras. And a marked similarity with CNTK. While CNTK allows for more compact representation, we present a slightly verbose illustration for ease of learning.\n",
    "\n",
    "Additionally, you will note that, CNTK model doesn't need to be compiled explicitly and is implicitly done when data is processed during training.\n",
    "\n",
    "CNTK effectively uses available memory on the system between minibatch execution. Thus the learning rates are stated as **rates per sample** instead of **rates per minibatch** (as with other toolkits)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Targetted reward\n",
    "REWARD_TARGET = 30 if isFast else 200\n",
    "# Averaged over these these many episodes\n",
    "BATCH_SIZE_BASELINE = 20 if isFast else 50\n",
    "\n",
    "H = 64 # hidden layer size\n",
    "\n",
    "class Brain:\n",
    "    def __init__(self):\n",
    "        self.params = {}\n",
    "        self.model, self.trainer, self.loss = self._create()\n",
    "        # self.model.load_weights(\"cartpole-basic.h5\")\n",
    "\n",
    "    def _create(self):\n",
    "        observation = C.sequence.input_variable(STATE_COUNT, np.float32, name=\"s\")\n",
    "        q_target = C.sequence.input_variable(ACTION_COUNT, np.float32, name=\"q\")\n",
    "\n",
    "        # Following a style similar to Keras\n",
    "        l1 = C.layers.Dense(H, activation=C.relu)\n",
    "        l2 = C.layers.Dense(ACTION_COUNT)\n",
    "        unbound_model = C.layers.Sequential([l1, l2])\n",
    "        model = unbound_model(observation)\n",
    "\n",
    "        self.params = dict(W1=l1.W, b1=l1.b, W2=l2.W, b2=l2.b)\n",
    "\n",
    "        # loss='mse'\n",
    "        loss = C.reduce_mean(C.square(model - q_target), axis=0)\n",
    "        meas = C.reduce_mean(C.square(model - q_target), axis=0)\n",
    "\n",
    "        # optimizer\n",
    "        lr = 0.00025\n",
    "        lr_schedule = C.learning_rate_schedule(lr, C.UnitType.minibatch)\n",
    "        learner = C.sgd(model.parameters, lr_schedule, gradient_clipping_threshold_per_sample=10)\n",
    "        trainer = C.Trainer(model, (loss, meas), learner)\n",
    "\n",
    "        # CNTK: return trainer and loss as well\n",
    "        return model, trainer, loss\n",
    "\n",
    "    def train(self, x, y, epoch=1, verbose=0):\n",
    "        #self.model.fit(x, y, batch_size=64, nb_epoch=epoch, verbose=verbose)\n",
    "        arguments = dict(zip(self.loss.arguments, [x,y]))\n",
    "        updated, results =self.trainer.train_minibatch(arguments, outputs=[self.loss.output])\n",
    "\n",
    "    def predict(self, s):\n",
    "        return self.model.eval([s])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Memory` class stores the different states, actions and rewards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Memory:   # stored as ( s, a, r, s_ )\n",
    "    samples = []\n",
    "\n",
    "    def __init__(self, capacity):\n",
    "        self.capacity = capacity\n",
    "\n",
    "    def add(self, sample):\n",
    "        self.samples.append(sample)\n",
    "\n",
    "        if len(self.samples) > self.capacity:\n",
    "            self.samples.pop(0)\n",
    "\n",
    "    def sample(self, n):\n",
    "        n = min(n, len(self.samples))\n",
    "        return random.sample(self.samples, n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Agent` uses the `Brain` and `Memory` to replay the past actions to choose optimal set of actions that maximize the rewards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "MEMORY_CAPACITY = 100000\n",
    "BATCH_SIZE = 64\n",
    "\n",
    "GAMMA = 0.99 # discount factor\n",
    "\n",
    "MAX_EPSILON = 1\n",
    "MIN_EPSILON = 0.01 # stay a bit curious even when getting old\n",
    "LAMBDA = 0.0001    # speed of decay\n",
    "\n",
    "class Agent:\n",
    "    steps = 0\n",
    "    epsilon = MAX_EPSILON\n",
    "\n",
    "    def __init__(self):\n",
    "        self.brain = Brain()\n",
    "        self.memory = Memory(MEMORY_CAPACITY)\n",
    "\n",
    "    def act(self, s):\n",
    "        if random.random() < self.epsilon:\n",
    "            return random.randint(0, ACTION_COUNT-1)\n",
    "        else:\n",
    "            return numpy.argmax(self.brain.predict(s))\n",
    "\n",
    "    def observe(self, sample):  # in (s, a, r, s_) format\n",
    "        self.memory.add(sample)\n",
    "\n",
    "        # slowly decrease Epsilon based on our eperience\n",
    "        self.steps += 1\n",
    "        self.epsilon = MIN_EPSILON + (MAX_EPSILON - MIN_EPSILON) * math.exp(-LAMBDA * self.steps)\n",
    "\n",
    "    def replay(self):\n",
    "        batch = self.memory.sample(BATCH_SIZE)\n",
    "        batchLen = len(batch)\n",
    "\n",
    "        no_state = numpy.zeros(STATE_COUNT)\n",
    "\n",
    "\n",
    "        # CNTK: explicitly setting to float32\n",
    "        states = numpy.array([ o[0] for o in batch ], dtype=np.float32)\n",
    "        states_ = numpy.array([(no_state if o[3] is None else o[3]) for o in batch ], dtype=np.float32)\n",
    "\n",
    "        p = agent.brain.predict(states)\n",
    "        p_ = agent.brain.predict(states_)\n",
    "\n",
    "        # CNTK: explicitly setting to float32\n",
    "        x = numpy.zeros((batchLen, STATE_COUNT)).astype(np.float32)\n",
    "        y = numpy.zeros((batchLen, ACTION_COUNT)).astype(np.float32)\n",
    "\n",
    "        for i in range(batchLen):\n",
    "            s, a, r, s_ = batch[i]\n",
    "\n",
    "            # CNTK: [0] because of sequence dimension\n",
    "            t = p[0][i]\n",
    "            if s_ is None:\n",
    "                t[a] = r\n",
    "            else:\n",
    "                t[a] = r + GAMMA * numpy.amax(p_[0][i])\n",
    "\n",
    "            x[i] = s\n",
    "            y[i] = t\n",
    "\n",
    "        self.brain.train(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "As any learning experiences, we expect to see the initial state of actions to be wild exploratory and over the iterations the system learns the range of actions that yield longer runs and collect more rewards. The tutorial below implements the [epsilon-greedy](https://en.wikipedia.org/wiki/Reinforcement_learning) approach (a.k.a. $\\epsilon$-greedy). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def plot_weights(weights, figsize=(7,5)):\n",
    "    '''Heat map of weights to see which neurons play which role'''\n",
    "    sns.set(style=\"white\")\n",
    "    f, ax = plt.subplots(len(weights), figsize=figsize)\n",
    "    cmap = sns.diverging_palette(220, 10, as_cmap=True)\n",
    "    \n",
    "    for i, data in enumerate(weights):\n",
    "        axi = ax if len(weights)==1 else ax[i]\n",
    "        if isinstance(data, tuple):\n",
    "            w, title = data\n",
    "            axi.set_title(title)\n",
    "        else:\n",
    "            w = data\n",
    "                \n",
    "        sns.heatmap(w.asarray(), cmap=cmap, square=True, center=True, #annot=True,\n",
    "                    linewidths=.5, cbar_kws={\"shrink\": .25}, ax=axi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploration - exploitation trade-off\n",
    "\n",
    "Note that the initial $\\epsilon$ is set to 1 which implies we are entirely exploring but as steps increase we reduce exploration and start leveraging the learnt space to collect rewards (a.k.a. exploitation) as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x21b66c1ad30>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAERCAYAAACD9ivUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl4VOXdxvHvTPY9ZEPTWqn6qm21LiAIRQrILsimDyK4\ngFgRRQsqgiKi6ItWBUuVgoCgVi2PQEAEWRRRobKIWrVvW2ytS6UlG9n3ZN4/ZuhEqkAkh5PJ3J/r\n6lVnznDmN79kcp/nLM/x+Hw+REREmsLrdgEiIhJ6FB4iItJkCg8REWkyhYeIiDSZwkNERJpM4SEi\nIk0W6fQbGGM6AQ9Za3sc8vwg4B6gFlhqrV3sdC0iItI8HB15GGPuABYBMYc8HwnMAXoB3YFfGGMy\nnaxFRESaj9O7rf4GDP2G538EfGKtLbHW1gLbgG4O1yIiIs3E0fCw1uYAdd+wKBkobvS4FEhxshYR\nEWk+bh0wL8EfIAclAUUu1SIiIk3k+AHzAM8hj/8MnGaMSQUq8O+yeuQo1qOJuEREvptD/w4fk+MV\nHj4AY8xIIMFau9gYMxnYhP8DLbbW/utoVlTXrh2Rn39O+bXXUjxrFnjD82zj7Oxs9u3b53YZLYJ6\nEaReBKkXQdnZ2c2+Tk+Izarr+/f775M+ahRRf/4zFUOHUjR3LkRFuV3XcacvRpB6EaReBKkXQYHw\naNaRR8httje0bUv+ihXUtG9PfE4Oadddh6ey0u2yRETCSsiFB4AvNZWC3/+equ7diX39ddKuvBJP\ncfGR/6GIiDSLkAwPAF98PIVLl1J56aXE7NpFxuWX483Lc7ssEZGwELLhAUB0NAeeeILy0aOJ+tOf\nyBg6lIh//tPtqkREWr3QDg+AiAiKH3qI0ptvJvIf/yBj8GAi9+51uyoRkVYt9MMDwOOhdNo0iu+5\nh4h//5v0YcOIev99t6sSEWm1Wkd4BJSPH8+Bxx7DW1xM+uWXE/PGG26XJCLSKrWq8ACovOIKDixa\nhMfnI+3aa4lbscLtkkREWp1WFx4AVf36UfDii/gSEmhz660kzp8PoXUxpIhIi9YqwwOgpmNH8nNy\nqD/xRJIffJDkmTOhocHtskREWoVWGx4AdWecQd6aNdSefjqJixfT5qaboLra7bJEREJeqw4PgIbv\nfY/8nByqO3Yk7uWXSb/qKjylpW6XJSIS0lp9eEBgOpMXXqCyXz9itm8nY/hwvLm5bpclIhKywiI8\nAIiL48BTTwWvRh88mIi//93tqkREQlL4hAf852r0kttvJ/KLL8gYMoSoDz5wuyoRkZATXuEB4PFQ\nNmkSRQ8/jLeoiPTLLiNm82a3qxIRCSnhFx4BFaNHU7hkCfh8pI0dS/wzz7hdkohIyAjb8ACo7tOH\nghUraEhLI/Wuu0h+4AFdCyIichTCOjwAas87j/yXX6b21FNJ/O1vaTNhAlRVuV2WiEiLFvbhAVB/\n8snkr1lDdadOxK1dS/oVV+ApLHS7LBGRFkvhEeBr04aCF16gYvBgYnbvJvPSS4n47DO3yxIRaZEU\nHo3FxlL0xBPBG0sNGkTUnj1uVyUi0uIoPA7l9VI6bRpFDz2Et6iIDGOIXb/e7apERFoUhce3qLjq\nKgqXLcPn9dLmF78gYdEit0sSEWkxIp1cuTHGA8wHzgGqgHHW2k8bLb8KuB0oAp6x1j7tZD1NVX3x\nxRSsWkXaNdeQMnMmEZ9/TsnMmRDpaNtERFo8p0ceQ4AYa20XYBow5+ACY0w6cD/QDegOjDLG/MDh\nepqs9uyzyV+7ltozzyRx6VLSxozRrLwiEvacDo+uwAYAa+1OoEOjZacAH1hri621PmA3cKHD9Xwn\n9d/7HvmrV1PVowexW7b4J1X88ku3yxIRcY3T4ZEMFDd6XGeMOfienwA/McZkGmPigYuBBIfr+c58\nSUkULltG2XXXEfXXv5JxySVE7d7tdlkiIq5weud9CZDU6LHXWtsAYK0tMsZMBlYCBcAeIP9IK8zO\nznaizqO3eDGcfz4Rt9xC5ogR8PTTcOWVrpTiei9aEPUiSL0IUi+c43R4bAcGAiuMMRcCHx1cYIyJ\nAM631nYzxkQDm4C7jrTCffv2OVXr0RsyhJjUVNqMH4931ChK332X0ttuA4/nuJWQnZ3dMnrRAqgX\nQepFkHoR5ESIOr3bKgeoNsZsBx4DJhljRhpjxllr6wGMMe8BbwDzrLUhMydIdffu5L/8MnU/+AFJ\nc+f658SqrHS7LBGR48Lj8/ncrqEpfC1tS8JbUECb664jZvduas47j8Knn6YhK8vx99VWVZB6EaRe\nBKkXQYGRR7PuGtFFgseoIT2dguXLqRg+nOj33yfjkkuI/L//c7ssERFHKTyaQ0wMRb/+NSV33knk\nvn1kDBlCzKZNblclIuIYhUdz8Xgou+UWChcuhPp60saOJXHePAit3YIiIkdF4dHMqgYOpGD1ahpO\nOIHkhx+mzYQJeHQgXURaGYWHA2rPPpu8V1+l+oILiHv5ZdKHDCHiq6/cLktEpNkoPBzSkJlJwfLl\nlI8cSfTHH5MxYADRu3a5XZaISLNQeDgpJobiRx6heNYsvAcOkG4M8S+84HZVIiLHTOHhNI+H8rFj\nKXj+eXwJCaTecQfJ06dDba3blYmIfGcKj+Ok5qKLyFu/ntozziBx6VLSr7wST2HIXFAvIvI1Co/j\nqP7kk8l/+WUq+/Yl5g9/IPOSS4j8y1/cLktEpMkUHseZLzGRA4sXU3rrrUR+8QUZgwbpHukiEnIU\nHm7weimdMoXCBQvA5yPt+utJmj0b6uvdrkxE5KgoPFxUNWgQ+WvXUteuHUlPPEHa6NE6DiIiIUHh\n4bK6H/2IvPXrqerZk9i33iJzwAAiP/7Y7bJERA5L4dEC+FJSKHzmGUonTybyyy/JHDyYuJUr3S5L\nRORbKTxaCq+X0ttuo2DpUnxRUbS55RaS77lH14OISIuk8Ghhqvv0CV4P8vTTpI8YgTc31+2yRES+\nRuHRAtWfcgr5a9dSOXAgMTt3ktm/P1Hvvut2WSIi/6HwaKF8CQkcWLCA4unT8ebmknHZZcQ/95zu\nDyIiLYLCoyXzeCi/8UYKXniBhsREUqdOJXXSJN0fRERcp/AIATUXXUT+hg3UnHsu8S+9RMagQbB3\nr9tliUgYU3iEiPrvf5/8Vasov/Zaov78Z+jQgdi1a90uS0TClMIjlMTEUPzggxx48kloaCBt/HiS\nZ8yAmhq3KxORMKPwCEGVQ4bA7t3U/s//kLhkCRnDh+PVbW5F5DiKdHLlxhgPMB84B6gCxllrP220\nfBQwGagDllprFzhZT6vyox+Rv24dKVOnEr9qFZl9+1L0xBNUd+/udmUiEgacHnkMAWKstV2AacCc\nQ5Y/AvQEugK3GWNSHK6nVfElJFA0bx5Fs2fjLS8nbfRokh59VLPziojjnA6PrsAGAGvtTqDDIcv/\nCLQB4gKPdRFDU3k8VFx9NfmrV1P/ve+RNHcu6aNG4S0ocLsyEWnFnA6PZKC40eM6Y0zj9/wTsAf4\nCHjFWlvicD2tVu0555C3YQNVvXoR8/bbZPbpQ/SuXW6XJSKtlKPHPIASIKnRY6+1tgHAGHM2cAlw\nMlAOPG+MGW6tPex0stnZ2U7VGnL+qxfZ2bBxIzz8MBHTp5Nx2WVw//0wdSp4W/e5Efq9CFIvgtQL\n5zgdHtuBgcAKY8yF+EcYBxUDFUC1tdZnjMnFvwvrsPbt2+dIoaEmOzv723txzTVEn3kmbSZMIOLu\nu6nesIED8+bRkJV1fIs8Tg7bizCjXgSpF0FOhKjTm6M5QLUxZjvwGDDJGDPSGDPOWvsF8BSwzRjz\nFpACLHO4nrBR06kTeZs3U3XxxcHdWG+95XZZItJKeHyhNdGeT1sSfke9VeXzkfDUUyTPng11dZRN\nnEjpbbdBpNODzuNHW5hB6kWQehEUGHl4mnOdrXtHuPgnV7zhBvJzcqg/6SSS5s0j/fLL8epLJSLH\nQOERJmrPO4+8jRv99wjZtYus3r2J2bTJ7bJEJEQpPMKILzmZAwsWUPTQQ3gqK0kfM4bkmTM1N5aI\nNJnCI9x4PFRcdRV5a9dSe+qpJC5aRMbQoUR89pnblYlICFF4hKm6n/yE/FdfpeLyy4n+4AMy+/Qh\nbsUK3alQRI6KwiOM+RISKHr8cQ7MmwceD21uvZXUm2/GU6IL/UXk8BQeQuXw4eRt2kTN+ecTv3o1\nmb17E717t9tliUgLpvAQAOpPPpn8Vaso/eUvidi3j/Rhw0h67DGoq3O7NBFpgRQeEhQVRekdd1Cw\nYgX1J55I0pw5ZAwfTsQXX7hdmYi0MAoP+S8HpzapvPRSot99138wPSfH7bJEpAVReMg38qWkcGD+\nfA7MnQv19bS5+WZSJ07EU1rqdmki0gIoPOTbeTxUGuM/mH7uuf7b3fbpQ9S777pdmYi4TOEhR1T/\nwx+Sv3o1pRMnEvHll2QMHUrSr34FtbVulyYiLlF4yNGJiqJ06lQKXnqJ+uxskn79azIGDSJy7163\nKxMRFyg8pElqOncm77XXqDCG6I8+IrNfPxIWLYKGBrdLE5HjSOEhTeZLSqJo7lwKFy+mISGBlJkz\nSb/iCiK++srt0kTkOFF4yHdW1b8/eVu2UNWrFzHbt5PZqxdxK1dqfiyRMKDwkGPSkJlJ4bJlFD3y\nCNTV0eaWW2gzfjyewkK3SxMRByk85Nh5PFRceSV5mzdTfcEFxL3yClm9ehGzZYvblYmIQxQe0mzq\n27WjYOVKSqZNw1tYSPpVV5EydSqe8nK3SxORZqbwkOYVEUHZzTeT98or1J55JgnPPUdmr15Ev/OO\n25WJSDNSeIgj6s46i7x16yi96SYi/vlPMi67jOTp0/FUVLhdmog0A4WHOCc2ltK77iJ/zRpqTzuN\nxKVL/aOQHTvcrkxEjpHCQxxXe/755G3cSNmNN/qnNxk+nOQZMzQKEQlhkU6u3BjjAeYD5wBVwDhr\n7aeBZW2B3wM+wAOcC9xprX3KyZrEJbGxlEyfTmX//qROmkTikiXEvv46RXPmUNOpk9vViUgTOT3y\nGALEWGu7ANOAOQcXWGv3W2t7WGt7BpbtARY5XI+4rLZ9e/8oZPx4Ij7/nPThw0m+9148lZVulyYi\nTeB0eHQFNgBYa3cCHb7ldb8BxltrdWlyOIiLo+See8hfvZr6H/6QxMWLdd90kRDjdHgkA8WNHtcZ\nY772nsaYQcDH1tq/OVyLtDC1HTqQu2kTZTfcQMRnn5E+dKh/FKJjISItnqPHPIASIKnRY6+19tDp\nV0cDjx/tCrOzs5ujrlah1fRiwQK4+moYM4bExYtJfP11eOop6NXrqFfRanrRDNSLIPXCOU6Hx3Zg\nILDCGHMh8NE3vKaDtfaoryDbt29fc9UW0rKzs1tXL9q1g/XrSZo7l8QFC/D07k3FiBEUz5iBLzX1\nsP+01fXiGKgXQepFkBMh6vRuqxyg2hizHXgMmGSMGWmMGQdgjMng67u1JJzFxfmvC1m3jtqf/IT4\n5cvJ6tGD2HXr3K5MRA7h8YXW9Nk+bUn4tfqtqtpaEhcuJGnOHDzV1VQOGEDxAw/Q0Lbtf7201fei\nCdSLIPUiKDDy8DTnOnWRoLRMUVGU3XwzuZs2Ud2pE3Hr15PVowfxL76o+4WItAAKD2nR6k87jYIV\nKyiaPRvq60m9/XbSR4wg4rPP3C5NJKwpPKTl83qpuPpqct94I3jXwosvJmHBAqirc7s6kbCk8JCQ\n0ZCdTeGyZRTOn48vPp6UWbPIGDQI9uxxuzSRsKPwkNDi8VA1eDB5b75JxfDhRH/4IXTs6J9osazM\n7epEwobCQ0JSQ1oaRfPmkb98OZx6KolLlpD1858Tu2GD26WJhAWFh4S0mq5d4cMPKZ08GW9hIWnX\nXUebMWOI+Oort0sTadUUHhL6YmMpve02cjdvprpzZ+I2bSKze3cSFi7UAXURhyg8pNWoP+00Cl56\niQNz5+KLiSHl/vvJHDCAqA8+cLs0kVZH4SGti8dDpTHkvfUWFSNGEPWnP5ExcKD//uklJW5XJ9Jq\nKDykVWpIS6NozhzyX3qJulNOIXHpUv88WWvW6Ap1kWZwVOFhjDnzkMcpzpQj0rxqunQhb/NmSm6/\nHe+BA6RNmED6FVcQ+TfdPkbkWBztyONnAMaY0YHHFztTjogDYmIomzSJ3C1bqOrZk5ht28js1Yuk\n//1fPOXlblcnEpKONjzONcakA/GBxxr3S8ipb9eOwmefpfDpp6lv25akJ5/0XxvyyivalSXSREcb\nHkuAPcC1xpg7gdOdK0nEQR4PVX37krd1K6W33oq3oIC0G24g7coridCuLJGjdlThYa39AP+uq5eA\n/cCjThYl4jRfXBylU6aQ+/rrVHXvTuxbb5HVqxdJs2frHuoiR0E3gwpRutFN0DH3wucjdsMGku+9\nl8ivvqIuO5uS++6jqn9/8DTr/XMcp9+LIPUiSDeDEnGCx0NV//7+XVkTJxKRl0fa9deTNnq0dmWJ\nfAuFh0iALz6e0qlTyX3tNaq6dSN261ayLr6Y5Pvv1wWGIodQeIgcov600yh84QUKFy2i/sQTSVy4\nkKyLLvLfArehwe3yRFoEhYfIN/F4qBowgNytWymZMgVPeTmpt99OxiWXEL17t9vVibhO4SFyOLGx\nlN16K7lvv03FsGFEf/ghGUOGkHrzzXh1MFbCmMJD5Cg0nHgiRb/5DXmrV1NzzjnE5+SQ1a0biY8/\nDpWVbpcnctwpPESaoPaCC8h/5RUOzJmDLzGR5Ece8U+4uG6drlKXsBLp5MqNMR5gPnAOUAWMs9Z+\n2mj5BcBjgYf/BkZba2ucrEnkmHm9VI4YQdWAAST9+tckLF5M2i9+QXWXLhTfdx91P/6x2xWKOM7p\nkccQIMZa2wWYBsw5ZPlTwLXW2m7ABuBkh+sRaTa+pCRKpk/3X6Xeqxcxf/gDmX36kHL77Xj373e7\nPBFHOR0eXfGHAtbanUCHgwuMMacDBcBkY8xWIM1a+4nD9Yg0u/pTT6XwmWcoeP556s44g4QXXySr\na1cS587VVCfSajkdHslAcaPHdcaYg++ZAXQG5gG9gF7GmO4O1yPimOru3cnbuJGiX/0KX3w8yY8+\nStZFFxFnra4PkVbH0WMeQAmQ1Oix11p78FtUAPzNWrsXwBizAf/IZOvhVhiYo0VQLxprUb244w4Y\nPx4efpiIxx6jzaRJtHn2WXjsMejRw/G3b1G9cJl64Rynw2M7MBBYYYy5EPio0bJPgURjzCmBg+gX\nAYuPtEJNdOanSd+CWmwvJkzAO3gwyQ8/TPzKldCzJ5V9+lBy993Un3aaI2/ZYnvhAvUiyIkQdXRW\n3UZnW/008NQYoD2QYK1dHNhN9XBg2R+stZOOsErNqhugL0ZQKPQi6o9/JPn++4nZsQNfZCTlV19N\n2aRJNKSlNev7hEIvjhf1IsiJWXU1JXuI0hcjKGR64fMRu3EjyQ88QOQ//kFDcjKlt9xC+ZgxEBvb\nLG8RMr04DtSLIE3JLhLKPB6q+vUjd8sWiu+7D7xeUh54wH9QfflyqK93u0KRo6bwEDneoqMpHzeO\n/du3U3rTTUQUFtJm8mQye/cmZvNmXakuIUHhIeISX2oqpXfdxf6336Z85EgiP/mE9GuvJX34cKLe\nfdft8kQOS+Eh4rKG7GyKH32UvNdfp7JvX2J27iRz8GDajBtHpO5kKC2UwkOkhag7/XQOPP00+Tk5\n1HToQNyrr5LZowcpU6bg/de/3C5P5GsUHiItTE3HjuSvXk3h009Td+qpJDz/PFldu5I0ezae4uIj\nr0DkOFB4iLREHg9VffuS99prFD36KL7UVJKeeIK2XbqQ8Nvf4tE9RMRlCg+RliwykoqRI8ndto2S\nu+6Chgb/6b1duhC/bBlUV7tdoYQphYdICPDFxVF2003sf+cdSm+5BU9ZGal3301Wt27E/f73UFfn\ndokSZhQeIiHEl5pK6Z13kvvOO5Rdfz0ReXm0ue02/90M16zR7L1y3Cg8REJQQ0YGJTNnsn/bNspH\njybiiy9ImzABzj+fmE2bdKGhOE7hIRLCGrKzKX74YXLffJOK4cPhww9JHzOGjEGDiH77bbfLk1ZM\n4SHSCtS3a0fRvHnw8cdUDhhA9Pvvk3HFFaRffjlRu3e7XZ60QgoPkdbkxz/mwKJF5L36KlU9e/rv\nqz5kCGmjRmnKE2lWCg+RVqj2pz+l8LnnyM/JobprV2K3biVz8GCFiDQbhYdIK1bTsSMFy5eTv2qV\nQkSalcJDJAzUdOqkEJFmpfAQCSMKEWkuCg+RMKQQkWOl8BAJY98aIiNHEr1jh9vlSQum8BCR/w6R\nt94iY/hw0ocOJeaNN3TFuvwXhYeI/MfBEMlbs4aqiy8mZtcu0kePJmPAAGLXr9fcWfIfCg8R+S+1\nHTpQ+Oyz5G7cSOXAgUR99BFp119P5sUXE7dqlWbxFYWHiHy7urPO4sDCheRt3UrF5ZcT+fe/02bi\nRLK6dSP++ed1P5Ew5vE5uC/TGOMB5gPnAFXAOGvtp42W/xIYB+QGnrrBWvvJYVbp27dvn1PlhpTs\n7GzUCz/1IsjpXkR8+SWJ8+cTv3w5nupq6k84gbIbb6Ri1Ch8cXGOve93od+LoOzsbABPc67T6ZHH\nECDGWtsFmAbMOWR5e+Aqa23PwP8OFxwi4rL6k06iePZs9r/zDmU33ICnuJiUe+8lq1MnEn/zG91j\nPYw4HR5dgQ0A1tqdQIdDlrcHphlj3jbGTHW4FhFpJg1t21IyYwa5u3ZReuuteGprSX7oIdp27Ejy\nrFl4//Uvt0sUhzkdHslA402ROmNM4/d8ERgP9AC6GmMGOFyPiDSjhrQ0SqdMYf/OnRRPn44vIYHE\nBQto27kzqZMmEbl3r9slikMiHV5/CZDU6LHXWtv4XL9fW2tLAIwx64DzgPWHW2Fg352gXjSmXgS5\n0ovsbJg1C6ZPh+efx/PII8RbS7y1MGgQTJkCXbu6UJZ+L5zidHhsBwYCK4wxFwIfHVxgjEkGPjbG\nnAlUAj2BJUdaoQ6A+elgYJB6EdQietGvH/TpQ+zmzSQ++STRa9fC2rXUdOhA2YQJVPXuDV7nT/Rs\nEb1oIZwIUad/gjlAtTFmO/AYMMkYM9IYMy4w4pgGbAXeBD621m5wuB4ROR68Xqr69iX/5ZfJX72a\nqt69iX73XdLGjiWze3fiX3xRp/mGOEdP1XWATtUN0FZVkHoR1JJ7Ebl3L4kLFhC3ahWe2lrq27al\nfNw4ykeNwpeS0uzv15J7cbyF4qm6IiIA1J1+OkVz5vhP8x0/Hk95OckPPkjbCy4gecYMIj7/3O0S\npQkUHiJyXDWceCIl99zD/l27KLn7bnxJSSQuWUJW1660uf56onft0kSMIUDhISKu8KWkUDZhAvt3\n7ODAk09Se/bZxK1fT8bQoWQMHEjsmjVQW+t2mfItFB4i4q6oKCqHDCF/3TryV62isn9/ov74R9Im\nTCCrSxcSfvtbXbneAik8RKRl8Hio6dSJA4sXk7ttG2Vjx+ItKiLlgQdo26GD/7jIZ5+5XaUEKDxE\npMWpb9eOklmz2L97t//K9ZSU4HGRceOI3rlTx0VcpvAQkRbLl5pK+Y03sv+dd/zHRX76U+JefZWM\nYcPI7NPHf71IZaXbZYYlhYeItHyNj4vk5FA5cCCRf/0rqbffzgkdOpD04INE/POfblcZVhQeIhI6\nPB5qOnbkwMKF7N+xg9KJE/F5vSTNn09W587+XVrbt2uX1nGg8BCRkNSQnU3p1Kns372bA3PnUnvW\nWf5dWsaQ2asXLFyIp6LC7TJbLYWHiIS22FgqjSF//Xry1qyhYvBgIv/2Nxg/3n+W1n336ep1Byg8\nRKR18Hio7dCBovnz2b9zJ8yYgS86msSnniLrZz8j7ZpriNm6FRoajrgqOTKFh4i0Og0nnAD33cf+\nnTs5MG8eteeeS+xrr5E+ahRZXbuSOH8+3sJCt8sMaQoPEWm9YmKoHD6c/FdeIW/dOipGjCBi/37/\nhIzt25M6cSLRu3frAPt3oPAQkbBQe+65FM2Zw7/37KH4vvuoO+kk4letImPIEDJ79yZ+2TI8paVu\nlxkyFB4iElZ8qamUjxtH3ptvkm+t/5qRTz4h9e67adu+PSl33knkn/7kdpktnsJDRMKTx0PNz37m\nv2Zk925KpkyhITWVhN/9jqw+fci49FLiXnoJqqrcrrRFUniISNhryMqi7NZbyX3nHQqWLaOqZ0+i\n3nuPNr/8JSe0b0/yzJlE7t3rdpktisJDROSgiAiqe/em8LnnyH3nHUpvvhlfZCSJixaR1aMHGYMH\nE7d8uS4+ROEhIvKN6k86idJp09i/ezeFTz1FVY8eRO3ZQ5vJk2l73nmk3HknUX/8Y9ieqaXwEBE5\nnOhoqi65hMLf/Y7cHTsonTyZhuRkEn73OzIHDCCzb1//mVphdsMqhYeIyFGq//73Kb3tNnJ37KDg\nueeoHDDAP7vv3Xdzwvnnk3rLLUTv2BEWo5FItwsQEQk5ERFU9+xJdc+eeHNziV+xgvgXXiB+5Uri\nV66k7pRTKL/ySiovu4yGzEy3q3WERh4iIsegISuLsgkTyH37bfJXrKBi2DAivvrqP7fPbTNmDLGv\nvgo1NW6X2qwcHXkYYzzAfOAcoAoYZ6399BtetxAosNbe5WQ9IiKO8Xio6dyZms6dKZ41i/hVq4hb\nvpy4TZuI27SJ+jZtqBw2jApjqDvrLLerPWZOjzyGADHW2i7ANGDOoS8wxtwAhH4nRUQCfKmplI8d\nS/7GjeRu2kTZ9deD1+u/D3vfvmT27k3CokV48/PdLvU7czo8ugIbAKy1O4EOjRcaYzoDFwALHa5D\nRMQVdT/5CSUzZ7J/zx4Kli6lsn9/IvfuJWXmTNq2b0+bsWOJ3bABamvdLrVJnA6PZKDx+Wt1xhgv\ngDHmBOBe4GbA43AdIiLuioqiuk8fDixezP733vNPznjGGcRt3EjaddfRtn17ku+9N2Tm1XL6bKsS\nIKnRY6+19uCdWC4H0oH1wIlAnDHmL9baZw+3wuzsbEcKDUXqRZB6EaReBLXYXmRnw9lnw4wZ8MEH\nsGwZEc+3ti12AAAIgUlEQVQ/T+LixSQuXgznngtXXQUjR8KJJ7pd7Tfy+Bw8H9kYMwwYaK0da4y5\nELjHWnvJN7zuGuCMozhg7tu3b58TpYac7Oxs1As/9SJIvQgKuV7U1BC7ZQtx1hL7+ut46urweb1U\nX3QRlcOGUdW/P76EhO+06kCINuseHqdHHjlAb2PM9sDjMcaYkUCCtXaxw+8tIhI6oqOp6tePqn79\n8BYUELt2LfErVxL75pvEvvkmDVOnUtWvH5XDhlHdrRtEunuZnqMjDwdo5BEQcltVDlIvgtSLoNbS\ni4hPPyU+J4e4VauI/OwzAOozMqgcPJjKYcOoPecc8Bx+UOHEyEMXCYqItGD1p5zinxJl2zbyXn6Z\n8muvhfp6EpcsIfOSS8j8+c9JfPxxIr744rjWpfAQEQkFHg+17dtT/OCD7H//fQqWLaPy0kuJ/Oor\nkh95hLadO5M+ZAjxzzyDt7DQ8XI0t5WISKiJiqK6d2+qe/fGU1pK7Pr1xK9aRfT27cTs3o1vxgyq\nu3WjcsgQqvr2daQEhYeISAjzJSVROWIElSNG4P3Xv4hbs4a4NWuI3bKF2C1baIiNhcrKZn9fhYeI\nSCvRcOKJlI8fT/n48UT8/e/+IFm3zpHjEzrmISLSCtWfeiplkyeT9/rrjqxf4SEiIk2m8BARkSZT\neIiISJMpPEREpMkUHiIi0mQKDxERaTKFh4iINJnCQ0REmkzhISIiTabwEBGRJlN4iIhIkyk8RESk\nyRQeIiLSZAoPERFpMoWHiIg0mcJDRESaTOEhIiJN5uhtaI0xHmA+cA5QBYyz1n7aaPlw4E6gAXjB\nWjvPyXpERKR5OD3yGALEWGu7ANOAOQcXGGO8wP8CPYEuwARjTJrD9YiISDNwOjy6AhsArLU7gQ4H\nF1hrG4AfWWvLgIxALTUO1yMiIs3A6fBIBoobPa4LjDgAf4AYY4YCHwBbgXKH6xERkWbgdHiUAEmN\n3y8w4vgPa22OtTYbiAGudrgeERFpBo4eMAe2AwOBFcaYC4GPDi4wxiQBa4E+1toa/KOOhm9cSyPZ\n2dkOlRp61Isg9SJIvQhSL5zj8fl8jq280dlWPw08NQZoDyRYaxcbY8YB4/Af6/gQmGitda4gERFp\nFo6Gh4iItE66SFBERJpM4SEiIk2m8BARkSZTeIiISJM5fapuszjSHFmthTEmEngaaAdEAw8C/wcs\nw38a88fW2psCr70e+AVQCzxorV1njIkFfgdk4b/G5hprbcFx/hjNyhiTBbwL9ALqCdNeGGOmApcC\nUfi/C28Rhr0IfEeewf8dqQOuJwx/L4wxnYCHrLU9jDGncoyfP3ApxeOB12621t5/pBpCZeTxrXNk\ntTKjgXxrbTegH/AE/s96l7X254DXGDPYGNMWmAh0DrxutjEmCrgR+DDw758D7nHjQzSXwB+KBUBF\n4Kmw7IUx5udA58Dvf3fgB4RpL4ABQIS19mfALPzz44VVL4wxdwCL8F9YDc3z+X8LXGGtvQjoZIw5\n50h1hEp4fOscWa2MJfjDjMC/ZXW+tfbtwHOvAr2BjsA2a22dtbYE+AT/qOw/fQq8ttfxKtwhj+L/\npd4HeAjfXvQFPjbGrAZeBl4hfHuxF4gM7I1Iwb+lHG69+BswtNHj9sfw+S8OXLAdba39LPD8Ro6i\nL6ESHoedI6u1sNZWWGvLAz/Ml4C78f/RPKgUfy+S+Ho/yvB/kRo/f/C1IckYcy2Qa63dTLAHjX/m\nYdML/BOHtgcuw7/l+Dzh24sy4IfAX4CFwDzC7Dtirc3Bv2F50LF8/oPPlRyyjpQj1REqf4CPOEdW\na2GMOQnYAjxjrf09X5+yJQkowt+P5EOeP8DX+3TwtaFqDNDbGPMG/i2mZ4HMRsvDqRcFwMbAVuRe\n/Mf9Gn+5w6kXk4AN1tozCP5eRDdaHk69OOhY/0YcGqJH1ZdQCY/t+Pd1cugcWa1JYD/lRmCKtfaZ\nwNPvG2O6Bf67P/A2sBvoaoyJNsakAGcCHwN/INCnwP+/TYiy1v7cWtvDWtsD/6zLVwGvhmMvgG34\n91tjjMkGEoDXA8dCILx6UUhwy7kI/0k/74dpLw5671i+F9baUqDaGPPDwO7AvhxFX0IlPHLwf7jt\nwGP4tz5ao2lAKnCPMeYNY8wWYDpwf+CzRwErrLX78Q/XtwGv4T9YVoP/+MBZxpi38c8Zdp8bH8JB\ntxOGvbDWrsP/B3IXsAb/rqvbgPvCrRf4zwhqb4x5C/9nnArcRHj24qDm+F6MB14AdgDvWWt3H+lN\nNbeViIg0WaiMPEREpAVReIiISJMpPEREpMkUHiIi0mQKDxERaTKFh4iINJnCQ+Q7MMZsNMakuV2H\niFsUHiLfTW+3CxBxky4SFDkMY0wCsBQ4Df8cQu/hnxLjWvzT5AwAfPinzz8J/xW+v7fWPmSMORl4\nE9iKfx4mgInW2m3H8SOIOEIjD5HDGwokWmvPxz/NtQ94ILCsu7X2K/z3RVhirb0A6IR/QsfLAq/5\nAfCqtfY8/NPPLDfGRBzXTyDiAI08RA7DGNMO/+jhU2AzkGOt/bMxpgH/VOlV+Gcq/ZDg1NgJ+O/N\nsgjYY63NaLS+L4DB1tr3j9uHEHGARh4ihxG4Qc5p+O9YlwS8ZowZjn8EAv6bdoH/Tn/nBUYYnQOv\nh6/fdwH837l6R4sWOQ4UHiKHYYwZDyyz1m621k4DNgFn4Q+AqMB01jvwz2yKMSYV/y0EBgdWkWWM\n6RNYNgiooZXeUkDCi3ZbiRyGMSYeWIL/gHc58DlwHfAUcD7+kCgDngROxn/A/AVr7azAAfM/AyuB\ns/Hfi/0Ga63CQ0KewkPEIYHw+Nham3TEF4uEGO22EnGWts6kVdLIQ0REmkwjDxERaTKFh4iINJnC\nQ0REmkzhISIiTabwEBGRJlN4iIhIk/0/EsCvL/Kgj48AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b66bd43c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def epsilon(steps):\n",
    "    return MIN_EPSILON + (MAX_EPSILON - MIN_EPSILON) * np.exp(-LAMBDA * steps)\n",
    "plt.plot(range(10000), [epsilon(x) for x in range(10000)], 'r')\n",
    "plt.xlabel('step');plt.ylabel('$\\epsilon$')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to train our agent using **DQN**. Note this would take anywhere between 2-10 min and we stop whenever the learner hits the average reward of 200 over past 50 batches. One would get better results if they could train the learner until say one hits a reward of 200 or higher for say larger number of runs. This is left as an exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode: 20, Average reward for episode 15.350000.\n",
      "Episode: 40, Average reward for episode 19.850000.\n",
      "Episode: 60, Average reward for episode 19.100000.\n",
      "Episode: 80, Average reward for episode 16.250000.\n",
      "Episode: 100, Average reward for episode 17.350000.\n",
      "Episode: 120, Average reward for episode 17.900000.\n",
      "Episode: 140, Average reward for episode 15.700000.\n",
      "Episode: 160, Average reward for episode 15.800000.\n",
      "Episode: 180, Average reward for episode 16.600000.\n",
      "Episode: 200, Average reward for episode 15.500000.\n",
      "Episode: 220, Average reward for episode 13.900000.\n",
      "Episode: 240, Average reward for episode 15.550000.\n",
      "Episode: 260, Average reward for episode 13.800000.\n",
      "Episode: 280, Average reward for episode 13.300000.\n",
      "Episode: 300, Average reward for episode 16.350000.\n",
      "Episode: 320, Average reward for episode 13.000000.\n",
      "Episode: 340, Average reward for episode 13.150000.\n",
      "Episode: 360, Average reward for episode 11.850000.\n",
      "Episode: 380, Average reward for episode 11.050000.\n",
      "Episode: 400, Average reward for episode 12.050000.\n",
      "Episode: 420, Average reward for episode 12.550000.\n",
      "Episode: 440, Average reward for episode 12.450000.\n",
      "Episode: 460, Average reward for episode 12.850000.\n",
      "Episode: 480, Average reward for episode 12.900000.\n",
      "Episode: 500, Average reward for episode 13.200000.\n",
      "Episode: 520, Average reward for episode 11.450000.\n",
      "Episode: 540, Average reward for episode 11.500000.\n",
      "Episode: 560, Average reward for episode 12.700000.\n",
      "Episode: 580, Average reward for episode 12.650000.\n",
      "Episode: 600, Average reward for episode 12.150000.\n",
      "Episode: 620, Average reward for episode 12.100000.\n",
      "Episode: 640, Average reward for episode 12.350000.\n",
      "Episode: 660, Average reward for episode 11.950000.\n",
      "Episode: 680, Average reward for episode 11.350000.\n",
      "Episode: 700, Average reward for episode 11.750000.\n",
      "Episode: 720, Average reward for episode 11.100000.\n",
      "Episode: 740, Average reward for episode 11.050000.\n",
      "Episode: 760, Average reward for episode 11.400000.\n",
      "Episode: 780, Average reward for episode 11.200000.\n",
      "Episode: 800, Average reward for episode 11.250000.\n",
      "Episode: 820, Average reward for episode 12.100000.\n",
      "Episode: 840, Average reward for episode 11.650000.\n",
      "Episode: 860, Average reward for episode 11.950000.\n",
      "Episode: 880, Average reward for episode 11.150000.\n",
      "Episode: 900, Average reward for episode 11.050000.\n",
      "Episode: 920, Average reward for episode 10.500000.\n",
      "Episode: 940, Average reward for episode 11.450000.\n",
      "Episode: 960, Average reward for episode 10.900000.\n",
      "Episode: 980, Average reward for episode 11.950000.\n",
      "Episode: 1000, Average reward for episode 11.050000.\n",
      "Episode: 1020, Average reward for episode 11.850000.\n",
      "Episode: 1040, Average reward for episode 13.000000.\n",
      "Episode: 1060, Average reward for episode 16.650000.\n",
      "Episode: 1080, Average reward for episode 17.000000.\n",
      "Episode: 1100, Average reward for episode 21.150000.\n",
      "Episode: 1120, Average reward for episode 26.050000.\n",
      "Episode: 1140, Average reward for episode 31.550000.\n",
      "Task solved in 1140 episodes\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAABUCAYAAACSjrEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXlYFMfa9u+BYTGAIEhcCKLRuCZqJG9iNCguuIGoICrK\nYsQYFDyo5AgIIiqbxIVoRDGe6BFRjHFBjRElKDlyXIi5NIrRxD1CYjRugCgM098f89Fnurump8lL\nUM77/K5rrmSm7nmquu+u6qKs6UfFcRwHgiAIgiAIgiBkMXneDSAIgiAIgiCIpgBNnAmCIAiCIAhC\nATRxJgiCIAiCIAgF0MSZIAiCIAiCIBRAE2eCIAiCIAiCUID6eTeAIAiCIAiCIJTy83vDBe9fO57X\naHXTxJkgCIIgCIJoOpiYPreqaeJMEARBEARBNBlU6uc3faU9zgRBvHB07doV3t7eGDt2LMaNG8f/\nt6yszOB3Lly4gIiIiAZrQ15eHgIDA+v1ndzcXIwZMwbjxo2Dv78/Lly4wJcdO3YM3t7eGDlyJObM\nmYPKykpFZXWUlpYiNDQUb775Jg4ePAgA2Lt3L9544w2kp6fj4cOHAICUlBQEBQXhhx9++DOHTRAE\n8cKjMjcTvBq1bsocSBDEi0a3bt1w8uRJ2NraPrc25OXlITs7G1u2bFGkv379OoKCgrB37144ODig\nsLAQCQkJOHr0KO7fvw8vLy/s2LEDzs7OWL58OSorK7Fo0SLZMjF79+7FmjVr8M033wAAKisr0b9/\nf3z11VdwcnICABw6dAhDhgyBmVnj3kwIgiAai2ve/oL3r+7b3mh104ozQRAvHBzHwdDf9KdPn8b4\n8eMRFhYGb29vTJw4EVevXsXp06cxevRoAMCTJ08QERGBcePGwcfHB/Hx8fz3d+zYgdGjR2Ps2LEI\nCQnBjRs3+LJPPvkEHh4emDBhAo4cOcJ/XlBQgAkTJsDHxweTJ0/G2bNnJe0yNzdHYmIiHBwcAACv\nv/467t27B41Gg6KiIvTs2RPOzs4AAH9/f+zfvx8AZMvEtGjRQvB+//79aNWqFR48eAAAuH37Nlq1\nakWTZoIg/qtRqdWCV2NCe5wJgnghCQoKgqmp7gcgHMfB2dkZa9asAQD8+OOPWLBgAfr06YOcnBzM\nnz8fUVFR/HePHDmCJ0+eYM+ePdBqtUhISMAvv/yC0tJSfP7559ixYwfs7OywZ88ehIWF4auvvkJ+\nfj7y8/Oxb98+WFhYYNasWQCAmzdvYtWqVdi6dStsbW1x5coVTJ06Ffn5+bC0tOTrdHJy4ld9Ad2W\niSFDhkCtVuPXX39F69at+bLWrVujsrISlZWVsmVWVlaCc2JnZ8f///fff48uXbrAwcGBnzh///33\n8Pb2/l+fe4IgiBeZ+mzP0Gg0WLBgAUpLS1FTU4PQ0FAMHjyYLy8oKEBGRgbUajV8fX3h5+cnG48m\nzgRBvJBkZWUZ3KrRpUsX9OnTBwDg6+uLJUuW4NGjR3y5q6sr0tPTERgYiP79+yM4OBjOzs7IycnB\nyJEj+QnouHHjkJycjNLSUpw8eRIeHh5o1qwZHzcrKwtFRUW4e/cupk6dyq+Cq9Vq3Lx5E126dJG0\nraqqClFRUfj999+xceNGADC4em5qaipbJqZuxbm6uhpnzpzBBx98ADs7Ozx48AAnTpxA3759AQDl\n5eU4ceIErl+/jg8//JAZnyAIoqmiMjdXrN23bx9atGiBtLQ0PHr0CGPHjuUnzhqNBqmpqdi9ezcs\nLCzg7++PIUOGwN7e3mA8mjgTBPFCIvfzC7XeP83V6UxM/rPz7JVXXsHhw4dx+vRpnDx5EsHBwVi4\ncCG0Wq0kllarhUajkdSpv9rdr18/rFy5ki/77bff0KpVK0mssrIyzJw5E506dcKWLVtg/v8H9zZt\n2uDcuXOC7zdv3hyWlpayZWJatGgBjuOQnZ2N8ePHA9CtQpeVlcHS0hIvv/wyAMDGxgY9evTATz/9\nZPAcEgRBNFXqsz1j5MiRGDFiBADdeK9//7h69SpcXFxgbW0NQLfoUlxcjOHDhzNjAbTHmSCIJsjF\nixf5SeGOHTvQp08f2NjY8OXbt29HdHQ0+vfvj8jISLi5ueHnn3+Gm5sbvv76a9y/fx8AsGvXLrRo\n0QIuLi5wc3PDoUOHUF5eDq1Wi9zcXABA3759UVRUhGvXrgEACgsLMWbMGDx79kzQpkePHiEgIADD\nhg3DihUr+EkzALz33nv44YcfcOvWLb7NQ4YMMVomxsbGBuXl5WjevDm/+tyiRQscP34cQ4cO/d+d\nVIIgiCZCfZ6q0axZM7z00kuoqKhAREQE5s6dy5dVVFQI7h1WVlYoLy+XjUcrzgRBvHCoVCrJHmeV\nSoW5c+fC0tISjo6OWLVqFW7fvo2WLVsiLS0Nt2/f5r8/duxYFBcXY9SoUWjWrBmcnJwQHBwMGxsb\nBAcHIzg4GIBu0pmZmQkAGDhwIH7++Wf4+vrC1tYWXbt2xYMHD9CxY0csWbIE8+bNA6BbiV63bp1k\nRXj79u24c+cO8vPz+R8WqlQqbN68Gfb29khOTsbs2bOh0Wjg7OyMtLQ0AJAtY9G3b1/4+vry7x0d\nHTFnzhzBijtBEMR/M/XZqgEAv/76K8LDwxEQEIBRo0bxn1tbW6OiooJ/X1lZiebNm8vXTY+jIwii\nKXH69GksXbrU4JMnCB2lpaXYs2cPwsPDn3dTCIIgGpTSeQsE751WJhvU3rt3D0FBQYiPj+d/B1KH\nRqOBp6cndu7cCUtLS0yaNAnr16/nt72xoBVngiCI/zIqKyuRl5eHCxcu4Oeff8Zrr732vJtEEATR\nYNRnxTkzMxOPHz9GRkYG1q5dC5VKhQkTJqCqqgp+fn6IiYnBtGnTwHEc/Pz8ZCfNAK04EwRBEARB\nEE2IXxcmCd63WRrbaHXTijNBEARBEATRZFCZPb/pK02cCYIgCIIgiCZDfX8c2JD85RPnVV8VypbP\n9RwIAEjbf1RWN3/0ICTuPiKrAYA4Hw+kH/zWqG7OqAHw/yRLVrM9IhAAMPbjf8jq9v49BAAwMf2f\nsrodc4Lhtewzo207EPUBhietl9XkxYYCAOJ2fC2rS5w4EgAQm3NQVpc0aRQ+zTtutG3hw99D8p58\nWc2CcbrHYik9b36rNsvqds6dilEpG4y27WDMDKw9XCSrCRvWHwAQmbVPVrciUJd9bdq6HFnd5zMn\nYfp6eQ0AbAydpPgYPFPldV9FzwAAzPrHl7K6jBDdc35DP9spq1v/gZ/ivhWcsU1W889Zk+vVNiXX\niNLzO37lJlnNl/PeB6DsfADKvFfaZ97P2C6r2TTLHwCMHuvG0EkAgL9vlb9+Pw7wNjo2ALrxYe4/\n9xrVrQoea/QaifPxAADFfVCJ90t2HTbatnjfYVh+4Jis5iMvdwDAop2HZHWL/XTPev0gc4es7rMP\nJxrV1OmUjpc+K+Sv392Ruus3Ze83srqYsUOM3j8A3T1k3PLPZTV7PpoGAEj4Mk9WlzBe98xbJWOX\n0vNmbKwBdOPNiORMWc2hBboEQBGb98jqPpk6DgAUjSN/27TbaNtWv++jyCtA2f0ZUHb9Ghu3AN3Y\npXS8fFFRmSnPHNjQ0IozQRAEQRAE0WSoTwKUhoYmzgRBEARBEESTQWXx/LZqKH5iPitVLUEQBEEQ\nBEE0JiozM8FLCefOnUNgYKDk882bN8PLywtBQUEICgrCjRs3ZOPIrjj/8ssvSElJwYULF6BWq6HV\natG5c2fExMSgQ4cOihpKEARBEARBEA1FfbdqbNy4Ebm5ubCyspKUlZSUIC0tDd27d1cUS7bm2NhY\nREZGolevXvxnZ8+eRUxMDHJyjG9AJwiCIAiCIIiGpL5P1XBxccHatWsxf/58SVlJSQkyMzNx9+5d\nuLu7Y8aMGbKxZLdqVFdXCybNANC7d+96NZYgCIIgCIIgGgqVuZngZQwPDw+Ympoyyzw9PbF48WJs\n2bIFZ86cQWGh/NPgZFecu3TpgpiYGLi5ucHGxgaVlZUoLCxEly5djDaSIAiCIAiCIBqahkyAEhwc\nDGtrawDAwIEDcfHiRQwcONCgXrbmhIQE5Ofn48yZM6ioqIC1tTUGDRoEDw+PBmswQRAEQRAEQSjl\nzz7HmeM4wfuKigp4eXnh66+/hqWlJU6ePInx48fL182JoxAEQRAEQRDEC0r5N8LtFDZDDK8Q11Fa\nWorIyEjk5OTgwIEDqKqqgp+fH/bt24ctW7bAwsIC7777LsLDw2Xj/OUT5xkbvpAt3zBjAgBlWZOU\nZgQ0Fqsu3tClGbKa/IWzAADL9hXI6qK8BwNQlm3KWCYhQJdNyFg2r1XBYwEoz86nJPOTsSxSgC6T\nlNLMa0qzeQWs2Sqr2zo7AJ98/S+jbYsY6aY4G6SSzGuAsmxTMdu/Mtq2FH9PxRnwlFxHALDuyL9l\ndTM9+gEAJqVvkdXlzAnCUgWZAxf6eCjOcKU0A968LbmyupVBY4xmywJ0GbOUXEcA4JG4TlZ3JG4m\nACjKqqY042L45/KZxj6d5gMACFqbLavbEjYFgLLr11gmU0CXzdRYrLp4SjO0GesPKf6eAJTdG+Zn\n7zfatrQpoxVn51N6XSrJ0KY0M6OxLHOr39d5r6QvADCagfRgzAxs+Oak0bbNGNJXcbbCNYfkM2TO\nHvEeAGX3SmP3LEB331KanW/Q4rWymqOLwgAo915JRj1j9wVAd29Qeq9U6oOSeYGxcRDQjYVTVsvf\nK7P/Jn1s24tERaEwQ6n1wP6NVjclQCEIgiAIgiCaDEp+EPhXQRNngiAIgiAIosnwZ/c4NwQ0cSYI\ngiAIgiCaDDRxJgiCIAiCIAgF0FYNgiAIgiAIglBAfTMHNiSyE+fAwEDU1NQIPuM4DiqVilJuEwRB\nEARBEI3On9mqce7cOSxfvhxZWcInihQUFCAjIwNqtRq+vr7w8/OTjSM7cf7oo48QFxeHtWvXGkxV\nSBAEQRAEQRCNhUpdvw0TGzduRG5uLqysrASfazQapKamYvfu3bCwsIC/vz+GDBkCe3t7g7FM5Crq\n1asXxowZg8uXL8PJyUnwIgiCIAiCIIjG5qmNleBlDBcXF6xdK33m99WrV+Hi4gJra2uYmZnB1dUV\nxcXFsrEocyBBEARBEATRZCgvLxe8t7GxMfod/cyBdZw5cwbZ2dlYuXIlAGD16tVo27atbNrtv/zH\ngcYy5cWMHQIARrMwJU4cibT9R43WN3/0IMUZBpVmwlKSQQyAouxV09YZ3xv++cxJirMcLdl1WFYX\n7zsMgLJsU0oz4Hmmysf6KnoGAOXZ+ZRk6VKaPc5YJrc4Hw8AMHotzR89CICybF5Ks5tFbTtgVLds\nspfibFNeyz6T1R2I+gAAjNa7bLKX4ixoSjJQAjCatW7HnGAAUHSskVnGM9utCPRWfN6UZAZTqlOa\nBU3JNQ4oGwcBYIWRrJyRXu5GzwegOyfGMksCuuySSvuz0rFreNJ6WV1ebKjR7KOALgOp0myQxu4N\nc0YNAAD4rdosq9s5d6rRzIeALvuhksydgLKscICy69LYfQHQ3RuU9hlj963PZ+o8VZK5VWkG1bWH\ni4zqwob1V9QXAOV9S8n9TekcQ+n5VTo+KLkulWY1VJoN8r8da2trVFRU8O8rKyvRvHlz2e/IbtUg\nCIIgCIIgiBeJahMzwUsp4k0WHTt2xM2bN/H48WNUV1ejuLgYvXv3lo1Bj6MjCIIgCIIgmgzVGs2f\n+p5KpQIAHDhwAFVVVfDz80NMTAymTZsGjuPg5+eHl19+WTZGvSfO1dXVMH+Oz88jCIIgCIIg/u+i\n0Wrr/R0nJyd+f7OXlxf/ubu7O9zd3RXHMbhVo6CgAIMGDYKHhwcOHjzIfz59+vR6N5YgCIIgCIIg\nGoLqmlrBqzExuOK8fv167N27F1qtFhEREXj27BnGjRsn2R9CEARBEARBEI1FTe2f26rREBicOJuZ\nmcHW1hYAkJGRgeDgYLRp04bfH0IQBEEQBEEQjY2mtv5bNRoKg1s1nJyckJKSgidPnsDa2hqffvop\nlixZgmvXrjVm+wiCIAiCIAiC55lGI3g1JgYToGg0Guzbtw8jR45Es2bNAAD37t1DZmYmYmNjG7WR\nBEEQBEEQBAEAJ6/cErzv26ldo9VNmQMJgiAIgiCIJsPxyzcE79/r0t6gluM4JCQk4PLlyzA3N0dS\nUhKcnZ358s2bN+PLL7+Evb09AGDJkiVo395wvL/8Oc7Gsn6tCNRlm1KScejTPPlMNwAQPvw9LDWS\nPQ4AFvp4KM66N3Pjl7K6ddN1qRmVZHJTmnGooTO0KcnSZSxWXTyl2fmUZhBTkglr2b4Co22L8h6s\nKEseAMVZI41dSwt9PBRntlOa+dJYBscUf08AyvoMAGwsOCWrmz74HaPZMQFdhkylGReVXG+Assyi\nSrNGBmdsk9X8c9ZkAMozgSrpWwu/MJ5xcemEkYrHGmPxlk7QZTdzT/hUVncsIdzouAXoxq5Bi9ca\n1R1dFIbM/BOymg+HvgsARrMp7v17CAAoyvan9PwqzQYZ/vluWd2n03x0MRX0+/czthtt26ZZ/or7\njNLMjEqy+Cn11Ng9NXy4Lnvcqq8KZXVzPQcCAHxWyPuwO/J9o9lYAV1GVqXeKz2/sTkHZXVJk0YB\nUHaNKO1bG745KauZMaQvABjNkhg2rD8AZZkDlWayVer9i0p9nuOcn5+P6upq5OTk4Ny5c0hJSUFG\nRgZfXlJSgrS0NHTv3l1RPEqAQhAEQRAEQTQZqmuVP4LuzJkzcHNzAwD06tULFy5cEJSXlJQgMzMT\nd+/ehbu7O2bMmCEbjybOBEEQBEEQRJOhRqP8qRoVFRWwsbHh36vVami1WpiY6J6P4enpiSlTpsDa\n2hphYWEoLCzEwIEDDcYz+FQNFk+fPkV1dXV9vkIQBEEQBEEQDUZNrUbwksPa2hqVlZX8e/1JMwAE\nBwfDzs4OarUaAwcOxMWLF2XjyU6cr1y5glmzZiEmJgb//ve/MWrUKIwaNQpHjxrfq0kQBEEQBEEQ\nDU2NplbwkqNPnz4oLNTt0z979iw6d+7Ml1VUVMDLywtVVVXgOA4nT55Ejx49ZOPJbtVYtGgRIiIi\nUFpair/97W/Iy8uDhYUFpk+fjkGDBik9PoIgCIIgCIJoEGrqscfZw8MDRUVFmDRJ90CClJQUHDhw\nAFVVVfDz88O8efMQGBgICwsLvPvuuxgwYIBsPNmJs1arxdtvvw0AOHXqFBwcHHRfUtPWaIIgCIIg\nCKLxqTayyqyPSqXC4sWLBZ916NCB/39vb294e3srjie7VaNDhw6IjY2FVqtFamoqAGDDhg1o2bKl\n4goIgiAIgiAIoqGoz1aNhkY2AYpWq0VBQQGGDh3Kf5abm4thw4bx2QQJgiAIgiAIorEQP4e6MZ87\nLbvnwsTERDBpBoAxY8b8pQ0iCIIgCIIgCEM09iqzPrRZmSAIgiAIgmgy1CcBSkNDE2eCIAiCIAii\nyUArzgRBEARBEAShgPo8jo7jOCQkJODy5cswNzdHUlISnJ2d+fKCggJkZGRArVbD19cXfn5+svFo\n4kwQBEEQBEE0GerzOLr8/HxUV1cjJycH586dQ0pKCjIyMgAAGo0Gqamp2L17NywsLODv748hQ4bA\n3t7eYLx6pdwmCIIgCIIgiOeJprZW8JLjzJkzcHNzAwD06tULFy5c4MuuXr0KFxcXWFtbw8zMDK6u\nriguLpaNRyvOBEEQBEEQRJOhPivOFRUVsLGx4d+r1WpotVqYmJhIyqysrFBeXi4bjybOBEEQBEEQ\nRJOhPhNna2trVFZW8u/rJs11ZRUVFXxZZWUlmjdvLhuv0SbOxjZn63Pu3DksX74cWVlZzHKNRoMF\nCxagtLQUNTU1CA0NxeDBgyU6rVaLuLg4XL9+HSYmJli8eDE6derEjPnHH3/A19cXmzZtEqRiFOPj\n4wNra2sAwCuvvILk5GSJZsOGDSgoKEBNTQ0mT54MX19fiWbPnj3YvXs3VCoVnj17hkuXLqGoqIiP\nrX+sUVFRKC0thVqtxtKlS5ntq66uRkxMDG7fvg1ra2ssWrQI7dq148v1z+mtW7cQHR0NExMTvPba\na1i0aBFTV0dKSgpeffVVTJw4kan78ccfkZiYCFNTU5ibmyMtLY3fH6Svu3LlCuLj4wEALi4uSEpK\ngomJCbPO/fv3Izs7Gzk5OQbr/PDDD9G+fXsAgL+/P0aOHCnR3b9/H3FxcSgvL0dtbS2WLVvGX3f6\nunnz5uHevXvgOA6lpaV48803sWLFCkmdCQkJUKvVaN++PZKSkphtKykpQUJCAiwsLNC1a1fExcUx\nr9lOnTpJfJC7tvV9YOnatm2LpUuX8j4kJydj+fLlAo2LiwsWLlwo8ECr1RqsU98HVp1t2rSR+ODh\n4SHR9e7dW+BDcnIyMjIyBJoDBw5IPFi2bBnzOBctWiTwgdW21q1bY9GiRQIfWGOCubm5xAe5saPO\nBz8/P4mmpqZG0hfs7OwkOgCSvgDAYJ36PrDaVlNTI/Bh4sSJKCoqEmjs7e0lfcHJyUkQKyEhAevW\nrZP48PHHH0vq1Gg0Eh9Ybauurpb4AAjHXFNTU4NjEmtsZo1J+rqnT58yxyR9TW1tLXM8MlQna0wS\n12loTNLX2draGhyT9HVr1qxhjkn6mmfPnkk8YLWtqqqK6YH4fhYaGsr0wdB9T+yDWBcUFCQYk9LS\n0jB9+nSBJiQkRDImmZiYGKxT7INYFxgYyPRBrPvoo4+YPoh1T58+xd27dwGA9+H69euS42T5II4V\nEBDA9EE8Z/if//kfiQ+G5hViD8S6Hj16SDywt7eX6Hr37s304UXF2PYMffr06YOjR49ixIgROHv2\nLDp37syXdezYETdv3sTjx49haWmJ4uJihISEyAfkGonDhw9z0dHRHMdx3NmzZ7mZM2cydZ999hnn\n5eXFTZw40WCsXbt2ccnJyRzHcdzDhw85d3d3pu7IkSPcggULOI7juFOnThmss6amhgsLC+OGDx/O\nXbt2zWC9z54948aNG2ewvK6e0NBQjuM4rrKykluzZo2snuM4bvHixdwXX3zBLMvPz+fmzJnDcRzH\nFRUVcbNnz2bqtm7dyi1cuJDjOI67du0aN23aNL5MfE5DQ0O54uJijuM4Lj4+njty5AhT98cff3DT\np0/nPDw8uJycHIPxAgICuEuXLnEcx3E5OTlcSkoKUzdr1izuu+++4ziO46Kjo7kjR44w/S4pKeGC\ng4MFn4l1X3zxBbdp0ybJeRDroqOjua+//prjOI47efIkd+zYMaaujkePHnFjx47l7t27J9GEhYVx\n3377LcdxHBcZGckdPXqUGcvHx4c7e/Ysx3Ecl56ezu3bt09wzT569Ihzd3dn+sC6tu/fvy/xgRVP\n7MO0adMkmrCwMIkHhvqT2AeWbufOnRIfWDqxD8nJyQb7sL4HrOMMDw/nCgsLBT6wdL6+vhIfWGMC\nyweWTtwfWBpWX2DpWD4YGq/EPrB04v7A0rD6gtwYqe8DSxceHi7pDywdqz+Ix1xDY5JYZ2hM0tdd\nvXqV6YM4Fms8YtXJ8oClMzQmiXWGxiRD9yF9H8QaQ2OS+HywPGDdz1g+sHQsH1g6sQ+JiYkSDcsH\nQ/dasQ8sHcsHlo7lg9w9vs6HsrIyiYblAysWywfWnEHsQ0ZGhkTD8oAVi9UXWDrWmPQiE7Q2W/CS\nQ6vVcvHx8dzEiRO5iRMncteuXeP279/Pz7uOHj3K+fr6cj4+Pty2bduM1t1of07Ibc7Wx8XFBWvX\nrpWNNXLkSERERADQrSqr1eyF86FDh2Lp0qUAdH8p2traMnXLli2Dv78/Xn75Zdl6L126hCdPniAk\nJARTp07FuXPnJJrjx4+jc+fOmDVrFmbOnIlBgwbJxjx//jyuXLli8PEn7du3R21tLTiOQ3l5OczM\nzJi6K1euYMCAAQCADh064Nq1a3yZ+JyWlJTgrbfeAgAMGDAAJ06cYOqePHmC2bNnw9vbW1CXWLdq\n1Sp06dIFgG6F3MLCgqn79NNP4erqiurqaty9exc2NjYSzYMHD5Ceno7Y2FjZOktKSnDs2DEEBAQg\nNjYWT548Yeq+//57/Pbbb3j//fdx4MABvPPOO0xdHatXr0ZAQAAcHBwkmm7duuHBgwfgOA6VlZX8\ndSfW3blzB7169QIAvPnmmzhz5ozgmq2trYWpqSkuXrwo8YF1bbN8EMdTq9VIT08X+NClSxeJhuUB\nq86HDx9KfGDpSkpKcPToUYEPYp2pqanEh9DQUIN9WN8D1nF269YNDx8+FPjAOr+//fabxAf9MaGs\nrAy2trZMH1hjR1VVlcAHlobVF1i6NWvWSHxg6Vg+sHRiH/r16yc5TlZfkBsj9X1g6Vj9QXx+mzdv\nzuwP+mMux3FMDwDp2Cz2oA59nUqlYvogjsXqC6w6WR6wdIbGJLHO0Jhk6D6k74NY0717d+aYJNax\nPGDdz1g+sHQsH1g6sQ/l5eUSDcsHViyWDywdyweWjuWD3D2+zoe7d+9KNCwfWLFYPojnDO7u7hIf\njhw5IplXsDxgzT9YfYGlY41JLzLVNbWClxwqlQqLFy9GTk4OcnJy0KFDB3h5efHzLnd3d3z55ZfY\ntWsX/P39jdbdaBNnQ5uzxXh4eMDU1FQ2VrNmzfDSSy+hoqICERERmDt3rkGtiYkJoqOjkZSUhNGj\nR0vKd+/eDQcHB/Tv3x8cx8nWa2lpiZCQEPzjH/9AQkICPvroI8kxPHjwABcuXMDq1auRkJCAyMhI\n2ZgbNmxAeHi4wXIrKyvcvn0bI0aMQHx8PAIDA5m6bt264dixYwCAs2fP4vfff+ePR3xO9Y9TfyO8\nWPfKK6+gZ8+ekrrEupYtWwLQ3RC2bduGqVOnMnUqlQplZWUYPXo0Hj58iK5duwo0df/MGx0djWbN\nmgnaKY7Vq1cvzJ8/H1u3boWzszPWrFnD1JWWlsLOzg6bNm1C69atsWHDBqYOAO7fv49Tp07Bx8eH\nqan7JzhPT0/cv38fb7/9NlPn7OyM7777DgBw9OhRVFVVMa9Zlg8snZOTk8QHls7BwUHgwwcffMDs\nJ2IPxLEKP0DLAAAJrElEQVQiIiIQGxsr8UGsmzNnDnr27ImoqCiBD6y2iX3Iyspitk3sAavOun9G\n1PeBVSfLB+A/Y0JiYiK8vLwM9gfx2MHyQawx1BfEOlZfEOs8PT2ZPrDi9erVS+KD+DgN9QXWGCn2\ngaVj+SA+v6NHj5b4cOnSJcmYqz+O1nnAGptZHrB0Yh9atmwp0bA8EMeqra1leiDWcRzHHJNYOpYP\nhu5D+j6wYrE8YMVi9QXW/YzVF1i6Nm3aSHxg6eq27NX5MH78eGadYh/EsebNm4fo6GiJD6w6e/To\nIfGBpbt165bEB0P3eH0fWBonJyeJDyyds7Mz/8SGOh/EcwbxvMLKygoVFRWSeQWrL7DmH6wxiaUz\nNCa9qGi0tYJXY9Joe5zlNmf/GX799VeEh4cjICAAo0aNktWmpqbijz/+gJ+fHw4ePAhLS0u+rG6f\ncVFRES5duoSoqCisW7eOn4To0759e7i4uPD/b2dnh7t376JVq1a8xs7ODh07doRarUaHDh1gYWGB\n+/fvM58JWF5ejhs3bvA3HBabN2+Gm5sb5s6dizt37iAoKAj79++Hubm5QOfr64urV69iypQp6NOn\nD3r06AGVSsWMqX/elWyEV8LBgweRmZmJDRs2oEWLFgZ1bdu2RV5eHnbu3ImUlBSkpqbyZSUlJbh1\n6xYSEhLw7NkzXL16FSkpKYiJiZHEGTp0KP+HmIeHBxITE5n12dnZ8av+gwcPRnp6usG2HTp0CF5e\nXgbPW1JSErZt24aOHTsiOzsbqamp/B5JfZKTk5GUlITa2lq4urryK/D616ynpyc+/vhj/jv6Pii9\ntlk6sQ8sDcsDfV27du0M+iA+hvLycqYPYl1KSorEB1bbWB6IY/Xr14/pg1jXvXt3pg/Af8aE8ePH\n49mzZ0wf9HWsscOQpqCggNkXxDpDfaFON3jwYDg6OhrsD/rxcnJy+NVFfR/0j7N58+YG+4K4bYb6\ngvi8GeoP+vHWrVuHjz/+mPehrKwMRUVFKCoqwuXLlxEVFYUHDx5IPFA6NhvSnTp1ivchKipKosnI\nyJB4cPv2bYHO29sbr7zyisSDkpISgS46OlrQtjoPzp8/z+vqjtXU1FTiw+nTp5nHkJeXx/ugf5x1\nsX788Ufk5uYKPPjpp58kbZs/fz7Wr18v6Aus+9nFixclPii579WVsXRnzpzhfXB0dMQbb7wh0Yh9\nWLJkiSBWWVkZTE1NJT5ERkZK6hwwYADftjofWG0rKyuT+FD3OxBx+7755hveB1as6OhoHDhwQOBD\ndHS0RBcZGYn169cjIyOD98HMzEwyZ7hz547AB2tra7i5uRmdVxiaf5w8eVIwJhnSyd2fXzSeZ+bA\nRltx7tOnDwoLCwFAsjmbhdzq77179xASEoK///3vGDdunEFdbm4uv6piYWEBExMTyWR969atyMrK\nQlZWFrp27Yply5YxJ80AsGvXLv5CunPnDiorK+Ho6CjQuLq64l//+hevefr0qcGJZHFxMfr27Wuw\n/QBga2vL/7jAxsYGGo2GuVJ//vx5vPvuu8jOzsbw4cMN/vAS0P3zXt1fvd9++y1cXV0F5cZW3sXk\n5uYiOzsbWVlZcHJyMqibOXMmbt68CUD3V7S+FxzH4Y033sD+/fuxZcsWrFy5Ep06dWJOmgEgJCQE\n58+fBwCcOHECPXr0YOpcXV356664uFjy41D9Yz1x4gS/3YWFnZ0d70WrVq3w+PFjpq6wsBArVqzA\npk2b8PDhQ/Tr1495zXbr1k3ig9Jrm6UT+8DSsDwQ63r27Mn0gRWP5QNLJ/ahbdu2zOMUe8CKxfKB\npWP5wBoTXn/9dZw+fVrgg5Kxg6XJy8uT9AWxTqVSISwsTOKDWOfo6IiDBw9KfGDFmz17Nn744Qf+\nHJqbm0va9tZbb/H/KlXXFwwdp9gHls7Ozg5WVlYCH1i6Y8eOCXxITEwUjLlpaWlwc3OT9AWlYzNL\nd/z4cYEP+ppu3bph2bJlWLhwocQDcayDBw/i8OHDEg9Y8WbOnCnwoEePHpJ4aWlpcHd3l/jA0jk4\nOAh8YGnatWsn6Qus83HhwgVJXxDfzyoqKtC/f39JX1By3wPY98dTp04JfGDVGR8fL/FBrOvQoQPy\n8vIkPrDizZo1S+IDSzd06FCJD4aOVd8HVqx27dpJ+gIr1rlz5yQ+iOcMVVVV6Nu3r8CHfv36KZpX\nsOYf3377rWRMYtUZGxtr8P78IlKtqRW8GpNGW3H28PBAUVERJk2aBED3S1A5DK36AUBmZiYeP36M\njIwMrF27FiqVChs3bpSswg4bNgwxMTEICAiARqNBbGysRKO0TgAYP348YmJiMHnyZJiYmCA5OVly\ncbm7u+O7777D+PHjwXEcFi1aZDDu9evXZSe4ABAcHIwFCxZgypQp0Gg0iIyMZK56ubi44JNPPsH6\n9evRvHlzwa+rxURFRWHhwoWoqalBx44dMWLECEG5sfOgj1arRXJyMtq2bYuwsDCoVCq8/fbbzO0n\nM2bMQHR0NMzNzdGsWTPBKnF96gSAhIQELF26FGZmZnB0dMSSJUuYuqioKMTFxWH79u2wsbHBihUr\nBOX69d64cUPWj6VLl2LOnDlQq9UwNzfn93KKcXFxQXBwMJo1a4Z33nkHAwYMQFJSkuSajY2NRWJi\nosCH5ORkRde2uA9otVpcuXJF4MPTp08lsebOnSvxQGl/YuliYmKQnJws8CE9PV2iW7ZsGWJjY3kf\n2rZtK9F89tlnEg9YdbJ8YOmmTZsm8aGqqkowJsTFxeHVV19FXFycwIenT58aHTvE48uCBQsQExMj\n6QshISGSWPb29hIfrKysFI1XrHGtTZs2WLJkCe9DUlISEhMTBcfZtWtXxMbGIicnh+8LarWaWafY\nB1addnZ2mDt3rsCHFi1aSM6JSqWS+CCmocYklUrFP7XF2Jj04YcfGhyP6mLVZxFh8eLFAg+MjUn6\nPhiq19iYlJiYKDsm1Z031phUU1MjuJ+lpqbyT4DR90Gj0Ri97wHS+2NSUhL/FJw6H1xdXVFeXs5r\n6uYBYh9sbW0ldbKuAXGdKSkpsLCwkPhgbm4uOVZHR0eJD5aWlpJ4JiYmAh9YdWq1WklfcHR0lBzD\ngwcPmH1Bf86QkJDAP+2mzofIyEisWLHC6LxCPP+Ij4/HvHnzmH1BPE+xsrKS7Q8vGvVJud3QqLj6\nLi8SBEEQBEEQxP9BXuy1eIIgCIIgCIJ4QaCJM0EQBEEQBEEogCbOBEEQBEEQBKEAmjgTBEEQBEEQ\nhAJo4kwQBEEQBEEQCqCJM0EQBEEQBEEogCbOBEEQBEEQBKGA/wfhniEQtl7OCAAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b6728fd68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAABUCAYAAACSjrEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXtcVNXe/z8Dw8UDCF7ICyKaHa8nNemUaSheyEgkEdBI\nLiamKBgqKiAoiNzEG6aikKVHJDFLvJBKIkpHHlHCn6SYlvcjlI8mJiAKw+znj3mYM3vvNXs2/Yjk\nPN/36zWvmlmf+a6192evtRfLNfur4DiOA0EQBEEQBEEQkhj92Q0gCIIgCIIgiLYATZwJgiAIgiAI\nQgY0cSYIgiAIgiAIGdDEmSAIgiAIgiBkQBNngiAIgiAIgpCB8s9uAEEQBEEQBEHI5ac3J/De//V0\nXqvVTRNngiAIgiAIou1gZPynVU0TZ4IgCIIgCKLNoFD+edNX2uNMEMRzR//+/eHu7o7JkyfDw8ND\n+9/Kykq937l06RJCQ0NbrA15eXnw8/P7Xd/Nz8+Ho6Mj77NTp07B3d0drq6uWLBgAWpra2WVNVFR\nUYGgoCC88sorOHLkCADgwIEDePnll5GamopHjx4BAJKSkuDv74/vv//+d7WdIAjieUdhasJ7tWrd\nlDmQIIjnjQEDBqC4uBjW1tZ/Whvy8vKQlZWFXbt2Net7t27dwuzZs/HgwQOcP38eAPDw4UO4ublh\n7969sLe3x9q1a1FbW4uYmBjJMiEHDhzApk2bcOLECQBAbW0tRo4cia+//hp2dnYAgGPHjmHcuHEw\nMWndmwlBEERrccPdh/f+xUN7Wq1uWnEmCOK5g+M46Pub/ty5c/Dy8kJwcDDc3d0xbdo0XL9+HefO\nncOkSZMAAE+ePEFoaCg8PDwwZcoUrFixQvv9vXv3YtKkSZg8eTICAwNx69YtbdnGjRvh4uKCqVOn\n4vjx49rPCwoKMHXqVEyZMgXvv/8+Lly4wGxbXV0dli5disjISN7nRUVFGDx4MOzt7QEAPj4+OHz4\nsMEyIR06dOC9P3z4MLp06YKqqioAwN27d9GlSxeaNBME8R+NQqnkvVoT2uNMEMRzib+/P4yNNT8A\n4TgO9vb22LRpEwDghx9+wLJlyzBs2DBkZ2dj6dKlCA8P1373+PHjePLkCXJycqBWqxEbG4t//etf\nqKiowGeffYa9e/fCxsYGOTk5CA4Oxtdff438/Hzk5+fj0KFDMDMzw7x58wAAt2/fxoYNG7B7925Y\nW1vj2rVrmDFjBvLz82Fubs5rc0xMDHx8fNC3b1/e5z///DO6du2qfd+1a1fU1taitrZWsszCwoIX\nx8bGRvv/58+fR79+/dCpUyftxPn8+fNwd3f/3eecIAiiLdDa2zN0oYkzQRDPJZmZmXq3avTr1w/D\nhg0DAHh6eiIuLg6//fabttzR0RGpqanw8/PDyJEjERAQAHt7e2RnZ8PV1VU7AfXw8EBiYiIqKipQ\nXFwMFxcXtGvXThs3MzMTRUVFuH//PmbMmKFdBVcqlbh9+zb69eunrTMrKwtKpRIeHh64e/cur736\nVs+NjY0ly4Q0rTjX19ejtLQUH374IWxsbFBVVYUzZ85g+PDhAIDq6mqcOXMGN2/exJw5c5jxCYIg\n2ioKU1PZWpVKhWXLlqGiogINDQ0ICgrC2LFjteUFBQVIS0uDUqmEp6cnvL29JePRxJkgiOcSqZ9f\nKHX+aa5JZ2T0751nPXr0wDfffINz586huLgYAQEBWL58OdRqtSiWWq2GSqUS1am72j1ixAisX79e\nW/bLL7+gS5cuvDgHDhzA06dP4eHhgfr6eu3/Z2RkoFu3bigrK+N9v3379jA3N5csE9KhQwdwHIes\nrCx4eXkB0KxCV1ZWwtzcHC+88AIAwMrKCoMGDcKPP/6o9xwSBEG0VZqzPePQoUPo0KEDUlJS8Ntv\nv2Hy5MnaibNKpUJycjL2798PMzMz+Pj4YNy4cejYsaPeeLTHmSCINsfly5e1k8K9e/di2LBhsLKy\n0pbv2bMHERERGDlyJMLCwuDk5ISffvoJTk5OOHr0KB4+fAgA+Oqrr9ChQwc4ODjAyckJx44dQ3V1\nNdRqNQ4ePAgAGD58OIqKinDjxg0AQGFhId599108e/aM16Z9+/bh8OHDyMnJQUZGBszMzJCTkwNb\nW1u8+eab+P7773Hnzh1tm8eNGwcAkmVCrKysUF1djfbt22tXnzt06IDTp09j/PjxLXJuCYIgnnea\n81QNV1dX7ROX1Go1b+Hl+vXrcHBwgKWlJUxMTODo6IiSkhLJeLTiTBDEc4dCoRDtcVYoFFi4cCHM\nzc1ha2uLDRs24O7du+jcuTNSUlJ42yMmT56MkpISvPPOO2jXrh3s7OwQEBAAKysrBAQEICAgAIBm\n0pmeng4AGD16NH766Sd4enrC2toa/fv3R1VVFfr06YO4uDgsWrQIgGYleuvWrcwVYeExNNGxY0ck\nJiZi/vz5UKlUsLe3R0pKisEyFsOHD4enp6f2va2tLRYsWMBbcScIgvhPpjlbNZq239XU1CA0NBQL\nFy7UltXU1PAWXSwsLFBdXS1dNz2OjiCItsS5c+ewatUqvU+eIDRUVFQgJycHISEhf3ZTCIIgWpSK\nRct47+3WJ0rqf/75Z4SEhMDX1xceHh7az69evYp169YhIyMDgOY5+I6Ojnjrrbf0xqIlCoIgiP8w\namtrkZeXh0uXLuGnn376s5tDEATRoihMTXkvKR48eIDAwEAsWbKEN2kGgD59+uD27dt4/Pgx6uvr\nUVJSgqFDh0rXTSvOBEEQBEEQRFvh5+UJvPfdVkXp1SYkJODo0aN48cUXtdv+pk6dirq6Onh7e+PU\nqVPYvHkzOI6Dl5cXfHx89MYCaOJMEARBEARBtCF+iVvNe991RbgeZctDPw4kCIIgCIIg2gzN+XFg\nS/OHT5w3HTstWT7/7TcBABu+LpTULZw4GqsPFRisL9x9LDbnSdcJACET3kRA2ueSmn/Mex8A4L1h\np6Ru38IZAAD/LVmSul3B0+G1fofBtn256AO4rf5EUpMb/iEAIO6rbyR1Kzw1G9xj9h2T1K30fhtb\nj/+XwbbNdRmBlMMnJTVLJ40BIP+8Tf84U1KX9ZEfPNZ+ZrBtOYtnIuNEsaRm9jhNgojwz3Mldavf\nd9PoM76Q1GXMnoq527802Lats7wwec2nBnUHlgQaPNacxTMBAAv/cUBStyFgMgDgox37JXUffzBF\ndt/6MH2vpOaTOdOa1bZpqf+Q1O1dECD7/PpslL6O9oT6AQBCPpM+H5tnTgEABH2yT1K37UNvpOef\nMdi2OePfkHUdya0TAKKyj0jqEt57B7Ff5hlsW6zXBCzZfcigbo2vu8FrJNxd81xUuX1QzviQmJNv\nsG3LPMbLun8AwKr9xyV1y6e4AADmfSp9zaUFehnUNOnknje516+c8dc9ZbvBth1aOktW/wOAeAPn\nLfp/z9uUddL3t/1hH8juz4bGGkAz3hgaVw8sCQQAg9f5Gl9Nxk05PoRlGu4z6/zcsTb3lKRmsZsz\nAHn3Z0De9WtoDAE044jc6+15RWFCmQMJgiAIgiAIwiDNSYDS0tDEmSAIgiAIgmgzKMz+vK0ash9H\nx0pVSxAEQRAEQRCticLEhPeSQ1lZGfz8xFtQdu7cCTc3N/j7+8Pf3x+3bt2SjCO54vyvf/0LSUlJ\nuHTpEpRKJdRqNfr27YvIyEj07t1bVkMJgiAIgiAIoqVo7laN7du34+DBg7CwsBCVlZeXIyUlBQMH\nDpQVS7LmqKgohIWFYciQIdrPLly4gMjISGRnZzer0QRBEARBEATx/0tzn6rh4OCALVu2YOnSpaKy\n8vJypKen4/79+3B2dsbs2bMlY0lu1aivr+dNmgEYzKhCEARBEARBEH8UClMT3ssQLi4uMDY2ZpZN\nnDgRK1euxK5du1BaWorCQumn9EiuOPfr1w+RkZFwcnKClZUVamtrUVhYiH79+hlsJEEQBEEQBEG0\nNAqTlnu2RUBAACwtLQEAo0ePxuXLlzF69Gi9esmaY2NjkZ+fj9LSUtTU1MDS0hJjxoyBi4tLizWY\nIAiCIAiCIOTye5/jLEyWXVNTAzc3Nxw9ehTm5uYoLi6Gl5eXdN2UcpsgCIIgCIJoK1Sf4G+nsBqn\nf4W4iYqKCoSFhSE7Oxu5ubmoq6uDt7c3Dh06hF27dsHMzAxvvPEGQkJCJOP84RNnOVnLAHnZejYe\n/afB+kJdnQzGaor3dmK6pObYsjkAIDv7j5xsU+sMxAKAMDdng5mJ1vlpshzJzc5nKAPXMo/xBrNI\nAZpMUqE7cyQ1G2d4aNpo4FjD/ve8zdwq/UPTz+a+1+LZICP3fC2pS/KZCABYtOugpG69/7tY/sVR\ng21bNdXVYPY4QJNBTm5mu52FJZK6GaP/DgB4L3WXpC57gT+SDpww2LbIyeMMHuuqqa4A5PUFQJ4P\nhrJlAZqMWXKuIwCys3LKyaom97zJzaT4QdoeSd2OeT4A5GUaM5SRE9Bk5Yzea/j6jZ/mionJGZKa\nryM0P6gxlLEw1msCAMgaRwxdH4DmGpGbOVBuZkY5mfIMZW0FNJlb5YwhgPxspnIy5f3j2+8Mti1g\n1KuyMv0BMJhVdq7LCADyMgDLzZ4rN6PluLg0Sc2JFfMAyB+TfDftltTtnu8ru22GsiRunaWpU+59\nXM41YmgcBDRjoZzjfJ6pKSzivbccPbLV6qYEKARBEARBEESbQc4PAv8oaOJMEARBEARBtBl+7x7n\nloAmzgRBEARBEESbgSbOBEEQBEEQBCED2qpBEARBEARBEDJobubAlkRy4uzn54eGhgbeZxzHQaFQ\nUMptgiAIgiAIotX5PVs1ysrKsHbtWmRm8p84VFBQgLS0NCiVSnh6esLb21syjuTEefHixYiOjsaW\nLVv0piokCIIgCIIgiNZCoWzehont27fj4MGDsLCw4H2uUqmQnJyM/fv3w8zMDD4+Phg3bhw6duyo\nN5aRVEVDhgzBu+++i6tXr8LOzo73IgiCIAiCIIjW5qmVBe9lCAcHB2zZskX0+fXr1+Hg4ABLS0uY\nmJjA0dERJSXSORIocyBBEARBEATRZqiurua9t7KyMvgd3cyBTZSWliIrKwvr168HAHz88cfo3r27\nZNrtP/zHgXKz7snJNmUoKxGgyUwkN8ucnAxMgLwMYgCQcvikpG7ppDEGM1cBmuxVcjMOyckICMjL\nNiU3A57H2s8kNTmLZwIAorKPSOoS3ntHE9NAlq7lU1wMZvICNNm8Vh8qkNSEu48FIC/DFSAvU5Oh\n4wQ0xyo3Q5vczIFys37JyfZnqP8Bmj4oJ6MaIC/7FgBZx2rIA0Djg9zzZiijXtZHfrJ1hjJ+AZqs\nX3KucQAGs9Gt8HwLAJB65FtJ3YJ3RhnM2gpoMrf6b8kyqNsVPF12tk25Y5d7ynZJ3aGlswweJ6A5\nVrlZDQ3dG0ImvAlAnveGMtEBmmx0hs7vruDpAGAwG90aX3fZbTN0XwA09wa5mX3lZlyUk7nV0PUB\naK6RjBPFBnWzxw03mFE41NUJgPyMlnLGVblzDLnnV+74IMd7Q1mHAU3mYbnZIP/TsbS0RE1NjfZ9\nbW0t2rdvL/kdya0aBEEQBEEQBPE8UW9kwnvJRbjJok+fPrh9+zYeP36M+vp6lJSUYOjQoZIx6HF0\nBEEQBEEQRJuhXqX6Xd9TKBQAgNzcXNTV1cHb2xuRkZGYOXMmOI6Dt7c3XnjhBckYzZ4419fXw/RP\nfH4eQRAEQRAE8X8XlVrd7O/Y2dlp9ze7ublpP3d2doazs7PsOHq3ahQUFGDMmDFwcXHBkSP/3sM5\na9asZjeWIAiCIAiCIFqC+oZG3qs10bvivG3bNhw4cABqtRqhoaF49uwZPDw8RPtDCIIgCIIgCKK1\naGj8fVs1WgK9E2cTExNYW1sDANLS0hAQEIBu3bpp94cQBEEQBEEQRGujamz+Vo2WQu9WDTs7OyQl\nJeHJkyewtLTE5s2bERcXhxs3brRm+wiCIAiCIAhCyzOVivdqTfQmQFGpVDh06BBcXV3Rrl07AMCD\nBw+Qnp6OqKioVm0kQRAEQRAEQQBA8bU7vPfDX+qpV8txHGJjY3H16lWYmpoiISEB9vb22vKdO3fi\nyy+/1KbZjouLQ69evfTGo8yBBEEQBEEQRJvh9NVbvPdv9uulV3v8+HEUFBQgKSkJZWVlSE9PR1pa\nmrZ8yZIl+OCDDzBw4EBZdf/hz3FemnVYsjxl+iQA8jIObS84a7C+WWNfl501SU6mPwBY+I8DkroN\nAZMByMs4ZCjLEaDJdJR04ISkJnLyOADyM7S9kySdWetI5GzZGcTkZo+Tm8nNZ6N0NqQ9oX6yM4gZ\nys4XP80VgLysZYC8bJByM9ttOmY429T8t9+UnXFRznkDgD3/9f8kdT4jXpGdAU9uli451xsAWde5\noaxagCaz1ofpeyU1n8yZBkB+JlA5Gd/ktk1uf5ab+dI5drOk7lRsCEJ35hhs28YZHhi/Ks2gLn/5\nPOz6Z6mkxt/JEQDgtV56HPxykSaj5YSEbZK6vKgg2edXbp2Ldh2U1K33fxeAvOty1rZsSQ0AbA96\nT1aWRwAtmplxzMotBtt2MiYYOwtLJDUzRv8dAGT3e0PjyL6FMwweJ6A5VrnZTOVmwJOTQRWAwX6z\ncYaH7L4lt898evKcpC5wzGsA5I35cjPUfnZKus6Zzq8ZjPNn0pznOJeWlsLJSXONDhkyBJcuXeKV\nl5eXIz09Hffv34ezszNmz54tGY8SoBAEQRAEQRBthvpG+Y+gq6mpgZWVlfa9UqmEWq2GkZHmZ34T\nJ07E9OnTYWlpieDgYBQWFmL06NF641HKbYIgCIIgCKLN0KBS815SWFpaora2Vvted9IMAAEBAbCx\nsYFSqcTo0aNx+fJlyXjNmjg/ffoU9fX1zfkKQRAEQRAEQbQYDY0q3kuKYcOGobCwEABw4cIF9O3b\nV1tWU1MDNzc31NXVgeM4FBcXY9CgQZLxJLdqXLt2DevXr4e1tTUmTZqE6OhoGBkZISoqCmPGjJF7\nfARBEARBEATRIjSo5G/VcHFxQVFREd577z0AQFJSEnJzc1FXVwdvb28sWrQIfn5+MDMzwxtvvIFR\no0ZJxpOcOMfExCA0NBQVFRX46KOPkJeXBzMzM8yaNYsmzgRBEARBEESr09CMPc4KhQIrV67kfda7\nd2/t/7u7u8Pd3V12PMmJs1qtxmuvaX5ZefbsWXTq1EnzJSX9ppAgCIIgCIJofeqbseLc0kjuce7d\nuzeioqKgVquRnJwMAMjIyEDnzp1bpXEEQRAEQRAEoUuDqpH3ak0kE6Co1WoUFBRg/Pjx2s8OHjyI\nt956S5tNkCAIgiAIgiBai815/LwIIRPebLW6JfdcGBkZ8SbNAPDuu+/+oQ0iCIIgCIIgCH209iqz\nLrRZmSAIgiAIgmgzNCcBSktDE2eCIAiCIAiizdCcFWeO4xAbG4urV6/C1NQUCQkJsLe315YXFBQg\nLS0NSqUSnp6e8Pb2loxHE2eCIAiCIAiizdCcx9Hl5+ejvr4e2dnZKCsrQ1JSEtLS0gAAKpUKycnJ\n2L9/P8zMzODj44Nx48ahY8eOeuNRym2CIAiCIAiizVCvauS9pCgtLYWTkxMAYMiQIbh06ZK27Pr1\n63BwcIClpSVMTEzg6OiIkpISyXi04kwQBEEQBEG0GVTNWHGuqamBlZWV9r1SqYRarYaRkZGozMLC\nAtXV1ZLxaOJMEARBEARBtBmakwDF0tIStbW12vdNk+amspqaGm1ZbW0t2rdvLxmPtmoQBEEQBEEQ\nbYbmbNUYNmwYCgsLAQAXLlxA3759tWV9+vTB7du38fjxY9TX16OkpARDhw6VjNdqK86GftWoS1lZ\nGdauXYvMzExmuUqlwrJly1BRUYGGhgYEBQVh7NixIp1arUZ0dDRu3rwJIyMjrFy5Ei+99BIz5q+/\n/gpPT0/s2LGDl8NcyJQpU2BpaQkA6NGjBxITE0WajIwMFBQUoKGhAe+//z48PT1FmpycHOzfvx8K\nhQLPnj3DlStXUFRUpI2te6zh4eGoqKiAUqnEqlWrmO2rr69HZGQk7t69C0tLS8TExKBnz57act1z\neufOHURERMDIyAh//etfERMTw9Q1kZSUhBdffBHTpk1j6n744QfEx8fD2NgYpqamSElJ0W6s19Vd\nu3YNK1asAAA4ODggISEBRkZGzDoPHz6MrKwsZGdn661zzpw56NWrFwDAx8cHrq6uIt3Dhw8RHR2N\n6upqNDY2YvXq1drrTle3aNEiPHjwABzHoaKiAq+88grWrVsnqjM2NhZKpRK9evVCQkICs23l5eWI\njY2FmZkZ+vfvj+joaOY1+9JLL4l8kLq2dX1g6bp3745Vq1ZpfUhMTMTatWt5GgcHByxfvpzngVqt\n1lunrg+sOrt16ybywcXFRaQbOnQoz4fExESkpaXxNLm5uSIPVq9ezTzOmJgYng+stnXt2hUxMTE8\nH1hjgqmpqcgHqbGjyQdvb2+RpqGhQdQXbGxsRDoAor4AQG+duj6w2tbQ0MDzYdq0aSgqKuJpOnbs\nKOoLdnZ2vFixsbHYunWryIc1a9aI6lSpVCIfWG2rr68X+QDwx1xjY2O9YxJrbGaNSbq6p0+fMsck\nXU1jYyNzPNJXJ2tMEtapb0zS1VlbW+sdk3R1mzZtYo5Juppnz56JPGC1ra6ujumB8H4WFBTE9EHf\nfU/og1Dn7+/PG5NSUlIwa9YsniYwMFA0JhkZGemtU+iDUOfn58f0QahbvHgx0weh7unTp7h//z4A\naH24efOm6DhZPghj+fr6Mn0Qzhn+/ve/i3zQN68QeiDUDRo0SORBx44dRbqhQ4cyfXheac5WDRcX\nFxQVFeG9994DoDlnubm5qKurg7e3NyIjIzFz5kxwHAdvb2+88MIL0gG5VuKbb77hIiIiOI7juAsX\nLnBz585l6j755BPOzc2NmzZtmt5YX331FZeYmMhxHMc9evSIc3Z2ZuqOHz/OLVu2jOM4jjt79qze\nOhsaGrjg4GBuwoQJ3I0bN/TW++zZM87Dw0NveVM9QUFBHMdxXG1tLbdp0yZJPcdx3MqVK7kvvviC\nWZafn88tWLCA4ziOKyoq4ubPn8/U7d69m1u+fDnHcRx348YNbubMmdoy4TkNCgriSkpKOI7juBUr\nVnDHjx9n6n799Vdu1qxZnIuLC5edna03nq+vL3flyhWO4zguOzubS0pKYurmzZvHfffddxzHcVxE\nRAR3/Phxpt/l5eVcQEAA7zOh7osvvuB27NghOg9CXUREBHf06FGO4ziuuLiYO3XqFFPXxG+//cZN\nnjyZe/DggUgTHBzMffvttxzHcVxYWBh38uRJZqwpU6ZwFy5c4DiO41JTU7lDhw7xrtnffvuNc3Z2\nZvrAurYfPnwo8oEVT+jDzJkzRZrg4GCRB/r6k9AHlm7fvn0iH1g6oQ+JiYl6+7CuB6zjDAkJ4QoL\nC3k+sHSenp4iH1hjAssHlk7YH1gaVl9g6Vg+6BuvhD6wdML+wNKw+oLUGKnrA0sXEhIi6g8sHas/\nCMdcfWOSUKdvTNLVXb9+nemDMBZrPGLVyfKApdM3Jgl1+sYkffchXR+EGn1jkvB8sDxg3c9YPrB0\nLB9YOqEP8fHxIg3LB333WqEPLB3LB5aO5YPUPb7Jh8rKSpGG5QMrFssH1pxB6ENaWppIw/KAFYvV\nF1g61pj0POO/JYv3ak1a7c8JqV816uLg4IAtW7ZIxnJ1dUVoaCgAzaqyUsleOB8/fjxWrVoFQPOX\norW1NVO3evVq+Pj4GPwr48qVK3jy5AkCAwMxY8YMlJWViTSnT59G3759MW/ePMydOxdjxoyRjHnx\n4kVcu3ZN73MDe/XqhcbGRnAch+rqapiYmDB1165dw6hRowAAvXv3xo0bN7RlwnNaXl6OV199FQAw\natQonDlzhql78uQJ5s+fD3d3d15dQt2GDRvQr18/AJoVcjMzM6Zu8+bNcHR0RH19Pe7fvw8rKyuR\npqqqCqmpqYiKipKss7y8HKdOnYKvry+ioqLw5MkTpu78+fP45Zdf8MEHHyA3Nxevv/46U9fExx9/\nDF9fX3Tq1EmkGTBgAKqqqsBxHGpra7XXnVB37949DBkyBADwyiuvoLS0lHfNNjY2wtjYGJcvXxb5\nwLq2WT4I4ymVSqSmpvJ86Nevn0jD8oBV56NHj0Q+sHTl5eU4efIkzwehztjYWORDUFCQ3j6s6wHr\nOAcMGIBHjx7xfGCd319++UXkg+6YUFlZCWtra6YPrLGjrq6O5wNLw+oLLN2mTZtEPrB0LB9YOqEP\nI0aMEB0nqy9IjZG6PrB0rP4gPL/t27dn9gfdMZfjOKYHgHhsFnrQhK5OoVAwfRDGYvUFVp0sD1g6\nfWOSUKdvTNJ3H9L1QagZOHAgc0wS6lgesO5nLB9YOpYPLJ3Qh+rqapGG5QMrFssHlo7lA0vH8kHq\nHt/kw/3790Ualg+sWCwfhHMGZ2dnkQ/Hjx8XzStYHrDmH6y+wNKxxqTnmfqGRt6rNWm1ibO+XzUK\ncXFxgbGxsWSsdu3a4S9/+QtqamoQGhqKhQsX6tUaGRkhIiICCQkJmDRpkqh8//796NSpE0aOHAmO\n4yTrNTc3R2BgID799FPExsZi8eLFomOoqqrCpUuX8PHHHyM2NhZhYWGSMTMyMhASEqK33MLCAnfv\n3sXbb7+NFStWwM/Pj6kbMGAATp06BUCzh+e///u/tccjPKe6x6n7C1KhrkePHhg8eLCoLqGuc+fO\nADQ3hM8//xwzZsxg6hQKBSorKzFp0iQ8evQI/fv352ma/pk3IiIC7dq147VTGGvIkCFYunQpdu/e\nDXt7e2zatImpq6iogI2NDXbs2IGuXbsiIyODqQOAhw8f4uzZs5gyZQpT0/RPcBMnTsTDhw/x2muv\nMXX29vb47rvvAAAnT55EXV0d85pl+cDS2dnZiXxg6Tp16sTz4cMPP2T2E6EHwlihoaGIiooS+SDU\nLViwAIMHD0Z4eDjPB1bbhD5kZmYy2yb0gFVn0z8j6vrAqpPlA/DvMSE+Ph5ubm56+4Nw7GD5INTo\n6wtCHas+maZPAAAJVElEQVQvCHUTJ05k+sCKN2TIEJEPwuPU1xdYY6TQB5aO5YPw/E6aNEnkw5Ur\nV0Rjru442uQBa2xmecDSCX3o3LmzSMPyQBirsbGR6YFQx3Ecc0xi6Vg+6LsP6frAisXygBWL1RdY\n9zNWX2DpunXrJvKBpWvastfkg5eXF7NOoQ/CWIsWLUJERITIB1adgwYNEvnA0t25c0fkg757vK4P\nLI2dnZ3IB5bO3t5e+6izJh+EcwbhvMLCwgI1NTWieQWrL7DmH6wxiaXTNyY9r6jUjbxXa9Jqe5yl\nftX4e/j5558REhICX19fvPPOO5La5ORk/Prrr/D29saRI0dgbm6uLWvaZ1xUVIQrV64gPDwcW7du\n1U5CdOnVqxccHBy0/29jY4P79++jS5cuWo2NjQ369OkDpVKJ3r17w8zMDA8fPmQ+TLu6uhq3bt3S\n3nBY7Ny5E05OTli4cCHu3bsHf39/HD58GKampjydp6cnrl+/junTp2PYsGEYNGgQFAoFM6bueZfz\nC1I5HDlyBOnp6cjIyECHDh306rp37468vDzs27cPSUlJSE5O1paVl5fjzp07iI2NxbNnz3D9+nUk\nJSUhMjJSFGf8+PHaP8RcXFwQHx/PrM/Gxka76j927FikpqbqbduxY8fg5uam97wlJCTg888/R58+\nfZCVlYXk5GTtHkldEhMTkZCQgMbGRjg6OmpX4HWv2YkTJ2LNmjXa7+j6IPfaZumEPrA0LA90dT17\n9tTrg/AYqqurmT4IdUlJSSIfWG1jeSCMNWLECKYPQt3AgQOZPgD/HhO8vLzw7Nkzpg+6OtbYoU9T\nUFDA7AtCnb6+0KQbO3YsbG1t9fYH3XjZ2dna1UVdH3SPs3379nr7grBt+vqC8Lzp6w+68bZu3Yo1\na9ZofaisrERRURGKiopw9epVhIeHo6qqSuSB3LFZn+7s2bNaH8LDw0WatLQ0kQd3797l6dzd3dGj\nRw+RB+Xl5TxdREQEr21NHly8eFGrazpWY2NjkQ/nzp1jHkNeXp7WB93jbIr1ww8/4ODBgzwPfvzx\nR1Hbli5dim3btvH6Aut+dvnyZZEPcu57TWUsXWlpqdYHW1tbvPzyyyKN0Ie4uDherMrKShgbG4t8\nCAsLE9U5atQobduafGC1rbKyUuRD0+9AhO07ceKE1gdWrIiICOTm5vJ8iIiIEOnCwsKwbds2pKWl\naX0wMTERzRnu3bvH88HS0hJOTk4G5xX65h/FxcW8MUmfTur+/LzRnMyBLU2rrThL/aqRhdTq74MH\nDxAYGIglS5bAw8NDr+7gwYPaVRUzMzMYGRmJJuu7d+9GZmYmMjMz0b9/f6xevZo5aQaAr776Snsh\n3bt3D7W1tbC1teVpHB0d8c9//lOrefr0qd6JZElJCYYPH663/QBgbW2t/XGBlZUVVCoVc6X+4sWL\neOONN5CVlYUJEybo/eEloPnnvaa/er/99ls4Ojryyg2tvAs5ePAgsrKykJmZCTs7O726uXPn4vbt\n2wA0f0XresFxHF5++WUcPnwYu3btwvr16/HSSy8xJ80AEBgYiIsXLwIAzpw5g0GDBjF1jo6O2uuu\npKRE9ONQ3WM9c+aMdrsLCxsbG60XXbp0wePHj5m6wsJCrFu3Djt27MCjR48wYsQI5jU7YMAAkQ9y\nr22WTugDS8PyQKgbPHgw0wdWPJYPLJ3Qh+7duzOPU+gBKxbLB5aO5QNrTPjb3/6Gc+fO8XyQM3aw\nNHl5eaK+INQpFAoEBweLfBDqbG1tceTIEZEPrHjz58/H999/rz2Hpqamora9+uqr2n+VauoL+o5T\n6ANLZ2NjAwsLC54PLN2pU6d4PsTHx/PG3JSUFDg5OYn6gtyxmaU7ffo0zwddzYABA7B69WosX75c\n5IEw1pEjR/DNN9+IPGDFmzt3Ls+DQYMGieKlpKTA2dlZ5ANL16lTJ54PLE3Pnj1FfYF1Pi5duiTq\nC8L7WU1NDUaOHCnqC3LuewD7/nj27FmeD6w6V6xYIfJBqOvduzfy8vJEPrDizZs3T+QDSzd+/HiR\nD/qOVdcHVqyePXuK+gIrVllZmcgH4Zyhrq4Ow4cP5/kwYsQIWfMK1vzj22+/FY1JrDqjoqL03p+f\nR5rzVI2WptVWnFm/apRC36ofAKSnp+Px48dIS0vDli1boFAosH37dtEq7FtvvYXIyEj4+vpCpVIh\nKipKpJFbJwB4eXkhMjIS77//PoyMjJCYmCi6uJydnfHdd9/By8sLHMchJiZGb9ybN29KTnABICAg\nAMuWLcP06dOhUqkQFhbGXPVycHDAxo0bsW3bNrRv357362oh4eHhWL58ORoaGtCnTx+8/fbbvHJD\n50EXtVqNxMREdO/eHcHBwVAoFHjttdeY209mz56NiIgImJqaol27drxV4ubUCQCxsbFYtWoVTExM\nYGtri7i4OKYuPDwc0dHR2LNnD6ysrLBu3TpeuW69t27dkvRj1apVWLBgAZRKJUxNTbV7OYU4ODgg\nICAA7dq1w+uvv45Ro0YhISFBdM1GRUUhPj6e50NiYqKsa1vYB9RqNa5du8bz4enTp6JYCxcuFHkg\ntz+xdJGRkUhMTOT5kJqaKtKtXr0aUVFRWh+6d+8u0nzyySciD1h1snxg6WbOnCnyoa6ujjcmREdH\n48UXX0R0dDTPh6dPnxocO4Tjy7JlyxAZGSnqC4GBgaJYHTt2FPlgYWEha7xijWvdunVDXFyc1oeE\nhATEx8fzjrN///6IiopCdna2ti8olUpmnUIfWHXa2Nhg4cKFPB86dOggOicKhULkg5CWGpMUCoX2\nqS2GxqQ5c+boHY+aYjVnEWHlypU8DwyNSbo+6KvX0JgUHx8vOSY1nTfWmNTQ0MC7nyUnJ2ufAKPr\ng0qlMnjfA8T3x4SEBO1TcJp8cHR0RHV1tVbTNA8Q+mBtbS2qk3UNCOtMSkqCmZmZyAdTU1PRsdra\n2op8MDc3F8UzMjLi+cCqU61Wi/qCra2t6BiqqqqYfUF3zhAbG6t92k2TD2FhYVi3bp3BeYVw/rFi\nxQosWrSI2ReE8xQLCwvJ/vC80ZyU2y2Ngmvu8iJBEARBEARB/B/k+V6LJwiCIAiCIIjnBJo4EwRB\nEARBEIQMaOJMEARBEARBEDKgiTNBEARBEARByIAmzgRBEARBEAQhA5o4EwRBEARBEIQMaOJMEARB\nEARBEDL4H3X7OfXa23ARAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b67574b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAABUCAYAAACSjrEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXlUFNfW9p+GZrqA4IATIhpzHeMQyU0cguKAQ0QUgaiR\nwYhxAi8qKiAoiAyKE4mKYkz0iiSYQUWJigNKIp8DIUuiGE2cFROvA0ZAFJqu949+qdtVdbq6yEuI\n3G//1uqlTT29z6l66uw6FKdrqziO40AQBEEQBEEQhCwmf3UHCIIgCIIgCKIxQBNngiAIgiAIglAA\nTZwJgiAIgiAIQgE0cSYIgiAIgiAIBdDEmSAIgiAIgiAUoP6rO0AQBEEQBEEQSvnl7ZGC938/ldNg\nbdPEmSAIgiAIgmg8mJj+ZU3TxJkgCIIgCIJoNKjUf930ldY4EwTx0tG1a1d4enpi/Pjx8PLy4v+9\nd++ewc9cvHgRoaGh9daHnJwc+Pv71+kzV65cgb+/P7y8vODj44Pi4mJ+28mTJ+Hp6YnRo0dj3rx5\nqKioULStlpKSEsyaNQuvv/46Dh48CADYt28fevbsiZSUFDx58gQAkJSUhICAAPz4449/ZLcJgiBe\nelTmZoJXg7ZNlQMJgnjZ6NatG86cOQM7O7u/rA85OTnIyMjAzp07FemfP3+O4cOHIykpCa6ursjN\nzcWaNWtw8OBBPH78GB4eHti9ezecnJywZs0aVFRUICYmRnabmH379mHDhg04fvw4AKCiogIDBw7E\nN998A0dHRwDA4cOHMWzYMJiZNezFhCAIoqG47jlZ8P6V/Z83WNu0VIMgiJcOjuNg6Hf6c+fOITk5\nGa1atcKdO3dgZWWFxMREPHr0CCtWrMCBAwfw7NkzREZG4vbt21CpVHjttdcQFxcHANi9ezd27doF\nU1NTNG/eHEuXLkWHDh0AAB9++CGys7PRtGlTtG/fnm8zNzcXW7ZsgUajgaWlJRYvXow+ffoI+nXq\n1Ck4OzvD1dUVADB06FC0a9cOAJCfn49evXrByckJADB58mSMGzcOMTExstvENG3aVPD+wIEDaNWq\nFUpLS+Ho6Ii7d++iVatWNGkmCOK/mr9yqQZNnAmCeCkJCAiAqanuCyAcx8HJyQkbNmwAAPz0009Y\nsmQJ+vbti8zMTCxevBjh4eH8Z48ePYpnz55h79690Gq1iI2NxZ07d1BSUoJPP/0Uu3fvhr29Pfbu\n3Yvg4GB88803OHbsGI4dO4b9+/fDwsICc+bMAQDcunUL69evx65du2BnZ4erV69i6tSpOHbsGCwt\nLfk2b968iebNmyMqKgqXL1+GnZ0dFi5cCAD49ddf0bp1a17bunVrVFRUoKKiQnabtbW14JjY29vz\n///hhx/QpUsXNG/eHKWlpfzPPD096+X4EwRBvKw09PIMfWjiTBDES0l6errBpRpdunRB3759AQDe\n3t6Ii4vD77//zm93cXFBSkoK/P39MXDgQAQGBsLJyQmZmZkYPXo0PwH18vJCYmIiSkpKcObMGbi7\nu8PKyoqPm56ejvz8fDx48ABTp07l74Kr1WrcunULXbp04dvUaDT47rvvsHPnTvTs2RPHjx/HjBkz\ncOLECYN3z01NTWW3iam941xVVYXCwkJ88MEHsLe3R2lpKU6fPo1+/foBAMrKynD69GncuHEDM2fO\nNHyQCYIgGiEqc3PFWo1GgyVLlqCkpATV1dWYNWsWhg4dym/Pzc1Famoq1Go1vL294evrKxuPJs4E\nQbyUyH39Qq33Z7panYnJf77r3K5dOxw5cgTnzp3DmTNnEBgYiKVLl0Kr1UpiabVaaDQaSZv6d7sH\nDBiAdevW8dt+++03tGrVShCnZcuW6NixI3r27AkAGDZsGKKjo3Hnzh20adMGRUVFgs83adIElpaW\nstvENG3aFBzHISMjAz4+PgB0d6Hv3bsHS0tLtGzZEgBga2uLHj164OeffzZ4DAmCIBordVmqsX//\nfjRt2hTJycn4/fffMX78eH7irNFosHLlSuzZswcWFhaYPHkyhg0bhmbNmhmMR0/VIAii0XHp0iV+\nUrh792707dsXtra2/PbPP/8cERERGDhwIMLCwuDq6opffvkFrq6uOHToEB4/fgwA+Prrr9G0aVN+\nbfLhw4dRVlYGrVaLrKwsAEC/fv2Qn5+P69evAwDy8vIwbtw4vHjxQtCnQYMGoaSkBJcuXQIAFBQU\nwMTEBO3atcPbb7+NH3/8Ebdv3+b7PGzYMACQ3SbG1tYWZWVlaNKkCX/3uWnTpjh16hSGDx/+fz+w\nBEEQjYC6PFVj9OjR/BOXtFqt4MbLtWvX4OzsDBsbG5iZmcHFxQUFBQWy8eiOM0EQLx0qlUqyxlml\nUmH+/PmwtLSEg4MD1q9fj7t376JFixZITk7G3bt3+c+PHz8eBQUFeOedd2BlZQVHR0cEBgbC1tYW\ngYGBCAwMBKCbdKalpQEABg8ejF9++QXe3t6ws7ND165dUVpaik6dOiEuLg4LFiwAoLsTvXnzZskd\n4RYtWmDTpk2IjY1FZWUlzM3NsXHjRpibm6NZs2ZITEzE3LlzodFo4OTkhOTkZACQ3caiX79+8Pb2\n5t87ODhg3rx5gjvuBEEQ/83UZalG7fK78vJyhIaGYv78+fy28vJywU0Xa2trlJWVybdNj6MjCKIx\nce7cOf7pGYRhSkpKsHfvXoSEhPzVXSEIgqhXShYsEbx3XJcoq//1118REhICPz8/eHl58T+/cuUK\n1q5di61btwLQPQffxcUFI0aMMBiLblEQBEH8l1FRUYGcnBxcvHgRv/zyy1/dHYIgiHpFZW4ueMnx\n8OFDBAUFYdGiRYJJMwB06tQJt27dwtOnT1FVVYWCggLJo0YlbdMdZ4IgCIIgCKKx8OvSBMH7Niui\nDGoTEhJw6NAhvPLKK/yyv3fffReVlZXw9fXFyZMnsXHjRnAcBx8fH0yePNlgLIAmzgRBEARBEEQj\n4re4VYL3rZeFG1DWP/TlQIIgCIIgCKLRUJcvB9Y3f/rEecPhU7Lb5456GwCw/ps8Wd38MYORfOCE\n0fYWjx2CTUfyjeqCRwzE+6nytc23z9Hdrp+UslNWlzkvAAAQmPqZrO5fc96D7/odRvv25fyp8Eze\nJqvZv3g6ACDu6yOyumXeugXusV/lyOpifUYi7dhpo32bObw/1mSflNUs9HADAExM+Zesbvc83ZMN\n/DbsktXtmuuHCWu3G+3bnrD3sfX4GVnNjGG6AhGRn38jq0uaPAYAMOvjL2V1Wz7wxZxPvjLat9Qg\nH3it+dSobu/CafBZJ7+vXy14HwCwYGeWrG5dwDgAwD+375HVffT+BMVja8bWL2Q1W2e8W6e+KRlb\ns7cZP76bp/tgykfpspqMf/oDUHY8ABhtd/N0H6PnG6A755ScR0rbBIDo3YdkdfETRxvNDYAuPyzO\nMP4ly+QpY7Fqf66sJtxT91xUpWNQSX5I2nfcaN8ixw9DysFvZTXz3hkEAIjfc1RWFz3BHQAQ8qn8\nObJx2gSjmlqdsbG1eOwQAFB8/irJv+NXf2K0b/sWBSm+tiXuPSarW+KlexSiktyl9LgZyzWALt8Y\ny6t7F04DAKPnefKUsQCU+bBo136jfVvt56n4Wqnk+gwoO3/rO1++rKjMqHIgQRAEQRAEQRilLgVQ\n6huaOBMEQRAEQRCNBpXFX7dUQ/Hj6FilagmCIAiCIAiiIVGZmQleSigqKoK/v3QJyo4dO+Dh4YGA\ngAAEBATg5s2bsnFk7zjfuXMHSUlJuHjxItRqNbRaLTp37ozIyEh07NhRUUcJgiAIgiAIor6o61KN\nbdu2ISsrC9bW1pJtxcXFSE5ORvfu3RXFkm05KioKYWFh6N27N/+z8+fPIzIyEpmZmXXqNEEQBEEQ\nBEH8X6nrUzWcnZ2xadMmLF68WLKtuLgYaWlpePDgAdzc3DBjxgzZWLJLNaqqqgSTZgBGK6oQBEEQ\nBEEQxJ+FytxM8DKGu7s7TE1NmdvGjBmD5cuXY+fOnSgsLERenvxT3mTvOHfp0gWRkZFwdXWFra0t\nKioqkJeXhy5duhjtJEEQBEEQBEHUNyqz+nu2RWBgIGxsbAAAgwcPxqVLlzB48GCDetmWY2NjcezY\nMRQWFqK8vBw2NjYYMmQI3N3d663DBEEQBEEQBKGUP/ocZ3Gx7PLycnh4eODQoUOwtLTEmTNn4OPj\nI982ldwmCIIgCIIgGgtlx4XLKWyHGb5DXEtJSQnCwsKQmZmJ7OxsVFZWwtfXF/v378fOnTthYWGB\n/v37IyQkRDbOnz5xVlqlS0kVmw8PfWe0vdDRrkYr0QG6anTu8ZtlNUejZwNQVqkJULavxiokAroq\niWHp8pWJ1vp7AlBenU9J1S9jVaQAXSUppVXhlB63aZvlv2j66exJ2JgjX4ESAEJGvq24GmRU5kFZ\nXcKkdwAA8/+1T1a3PnA8Yr48bLRvy31H4YO03UZ1H8+ciNAde2U1H071AgDs/K5QVhfg6gJAWTUv\npZUDlVSsA6B4H5T4YKxaFqCrmDV9i/x5tG3WJAAwWlVt36IgAMDkD+Vz0ueh/kYrqgG6qmpKx4yS\n6qOAskpjSvOgsVi18UYlpslqDi+ZCQBYYcSvpf9bnU/JMTF2fgC6c8TYtSF0tCsA5ZUZlVRoU3pe\nKvVeaTVTJedvRv4PRvs2ZWBfxRVqlVaDVFLB0Vg+AnQ5SWlFS7fYjbKak7G6SZCSapAAELApQ1a3\nM3iK4r4prRhqrKLwl/OnAlB2jhjLg4AuFyrJby8z5XnCCtE2gwc2WNtUAIUgCIIgCIJoNCj5QuCf\nBU2cCYIgCIIgiEbDH13jXB/QxJkgCIIgCIJoNNDEmSAIgiAIgiAUQEs1CIIgCIIgCEIBda0cWJ/I\nTpz9/f1RXV0t+BnHcVCpVFRymyAIgiAIgmhw/shSjaKiIqxZswbp6cIniuTm5iI1NRVqtRre3t7w\n9fWVjSM7cV64cCGio6OxadMmg6UKCYIgCIIgCKKhUKnrtmBi27ZtyMrKgrW1teDnGo0GK1euxJ49\ne2BhYYHJkydj2LBhaNasmcFYJnIN9e7dG+PGjcOVK1fg6OgoeBEEQRAEQRBEQ/Pc1lrwMoazszM2\nbdok+fm1a9fg7OwMGxsbmJmZwcXFBQUFBbKxqHIgQRAEQRAE0WgoKysTvLe1tTX6Gf3KgbUUFhYi\nIyMD69atAwB89NFHaNu2rWzZ7T/9y4FrjVSPC/vf6nFxXx+R1S3zHmG0KhGgq0yktMqc15pPZTV7\nF04DoKyCGACj1dcWjx1itHIVoKteNecTeV1qkM7UpH3HZXWR44cBACasla/WtCfsfcUV8JTEApRX\n56vPKl1KPABgtILj/DG68p1KKjUZq6YH6CrqLf3CuG7Fu6MVV9tUWm3KmK/LfUcZHX+Abgwaq5S3\nxGs4AGXVtwBl1TaNeQDofFB63IxV1Ns110+xzljlTkBXvVPJOQ4or7qnpFKeseqNgK6Co7FqhYCu\nYqHS8ay0Op+SCnhKc77Hqo9lNdnhHwCA0WtDyMi3ASjz3lglOkBXjU5pNUhj1eiSp4xV3Ddj1wVA\nd21QWuFTqadKKrcau7YBuuubsWqFgK5i4YbD8p7OHaXzVMkcA1BWaXXTkXxZDQAEjxio+PgqzQ9K\n8uqiXfJVhwFgtZ8nNh/9f7Ka2e4DjMb5b8DGxgbl5eX8+4qKCjRp0kT2M7JLNQiCIAiCIAjiZaLK\nxEzwUop4kUWnTp1w69YtPH36FFVVVSgoKECfPn1kY9Dj6AiCIAiCIIhGQ5VG84c+p1KpAADZ2dmo\nrKyEr68vIiMjMW3aNHAcB19fX7Rs2VI2Rp0nzlVVVTD/C5+fRxAEQRAEQfz/i0arrfNnHB0d+fXN\nHh4e/M/d3Nzg5uamOI7BpRq5ubkYMmQI3N3dcfDgf9a2TZ8+vc6dJQiCIAiCIIj6oKq6RvBqSAze\ncd6yZQv27dsHrVaL0NBQvHjxAl5eXpL1IQRBEARBEATRUFTX/LGlGvWBwYmzmZkZ7OzsAACpqakI\nDAxEmzZt+PUhBEEQBEEQBNHQaGrqvlSjvjC4VMPR0RFJSUl49uwZbGxssHHjRsTFxeH69esN2T+C\nIAiCIAiC4Hmh0QheDYnBAigajQb79+/H6NGjYWVlBQB4+PAh0tLSEBUV1aCdJAiCIAiCIAgAOHP1\ntuB9v1fbG9RyHIfY2FhcuXIF5ubmSEhIgJOTE799x44d+Oqrr/gy23FxcejQoYPBeFQ5kCAIgiAI\ngmg0nLpyU/D+7S4dDGqPHj2K3NxcJCUloaioCGlpaUhNTeW3L1q0CO+//z66d++uqO0//TnOSiqv\nAVBUXenTk+eMtjfN7U3FVeaMVaWa984gAEBYunwlnrX+ngCUVeczVvEL0FX9UloBT2mFtneStsrq\nDkbOUFxBTGmVo/n/2ierWx84HoCySlhKj5uSKnmAsqplgLIKg0or26UdO21UN3N4f8UV2ial7JTV\nZc4LAABkfV8sqxv3Rg+jVQgBXSVCYxWzgkcMBACMSkyT1R1eMhOAssqXxqoVArqKhTO2fiGr2Trj\nXQAwWu1v97xAAMpyktIKbav258pqwj2HAlBeadUtdqOs7mRsCBbszDLat3UB4zAsLtWo7viyOfji\nTJGs5t1+vQEoy4MAMDJhi6wuJ2qWYu+VVHsDYLSq2mo/XS5X4te0zZmyGgD4dPYkxdUKlY57YxUL\nN06bYPT8AHTniFJPlVSqBJRVM1VaaTX2qxyjulifkdiRVyCrmTr4HwBgtHLrindHA1B23TKmqdXt\nPn1eVjOxv67Qxr++/V5WFzjoDQDKqhgrrVCbkf+DrGbKwL5G4/yV1OU5zoWFhXB11Z2jvXv3xsWL\nFwXbi4uLkZaWhgcPHsDNzQ0zZsyQjUcFUAiCIAiCIIhGQ1WN8kfQlZeXw9bWln+vVquh1WphYqL7\nmt+YMWMwZcoU2NjYIDg4GHl5eRg8eLDBeFRymyAIgiAIgmg0VGu0gpccNjY2qKio4N/rT5oBIDAw\nEPb29lCr1Rg8eDAuXbokG69OE+fnz5+jqqqqLh8hCIIgCIIgiHqjukYjeMnRt29f5OXpll2eP38e\nnTt35reVl5fDw8MDlZWV4DgOZ86cQY8ePWTjyS7VuHr1KtatWwc7OzuMHTsW0dHRMDExQVRUFIYM\nGaJ0/wiCIAiCIAiiXqjWKF+q4e7ujvz8fEyaNAkAkJSUhOzsbFRWVsLX1xcLFiyAv78/LCws0L9/\nfwwaNEg2nuzEOSYmBqGhoSgpKcE///lP5OTkwMLCAtOnT6eJM0EQBEEQBNHgVNdhjbNKpcLy5csF\nP+vYsSP/f09PT3h6eiqOJztx1mq1ePPNNwEAZ8+eRfPmzXUfUtN3CgmCIAiCIIiGp6oOd5zrG9k1\nzh07dkRUVBS0Wi1WrlwJANi6dStatGjRIJ0jCIIgCIIgCH2qNTWCV0MiWwBFq9UiNzcXw4cP53+W\nlZWFESNG8NUECYIgCIIgCKKhED8fvfZ56A2B7JoLExMTwaQZAMaNG/endoggCIIgCIIgDNHQd5n1\nocXKBEEQBEEQRKOhLgVQOI5DbGwsrly5AnNzcyQkJMDJyYnfnpubi9TUVKjVanh7e8PX11c2Hk2c\nCYIgCIIgiEZDXe44Hzt2DFVVVcjMzERRURGSkpKQmpoKANBoNFi5ciX27NkDCwsLTJ48GcOGDUOz\nZs0MxqPKgQRBEARBEESjobqmRvCSo7CwEK6urgCA3r174+LFi/y2a9euwdnZGTY2NjAzM4OLiwsK\nCgpk49EdZ4IgCIIgCKLRUJfH0ZWXl8PW1pZ/r1ar+bLb4m3W1tYoKyuTjUcTZ4IgCIIgCKLRoKnD\nGmcbGxtUVFTw72snzbXbysvL+W0VFRVo0qSJbDxaqkEQBEEQBEE0Gqo0NYKXHH379kVeXh4A4Pz5\n8+jcuTO/rVOnTrh16xaePn2KqqoqFBQUoE+fPrLx6I4zQRAEQRAE0Wioy1INd3d35OfnY9KkSQCA\npKQkZGdno7KyEr6+voiMjMS0adPAcRx8fX3RsmVL2XiyBVDqE2OPA9GnqKgIa9asQXp6OnO7RqPB\nkiVLUFJSgurqasyaNQtDhw6V6LRaLaKjo3Hjxg2YmJhg+fLlePXVV5kxHz16BG9vb2zfvl1Qw1zM\nhAkTYGNjAwBo164dEhMTJZqtW7ciNzcX1dXVeO+99+Dt7S3R7N27F3v27IFKpcKLFy9w+fJl5Ofn\n87H19zU8PBwlJSVQq9VYsWIFs39VVVWIjIzE3bt3YWNjg5iYGLRv357frn9Mb9++jYiICJiYmODv\nf/87YmJimLpakpKS8Morr2DixIlM3U8//YT4+HiYmprC3NwcycnJ/DdS9XVXr17FsmXLAADOzs5I\nSEiAiYkJs80DBw4gIyMDmZmZBtucOXMmOnToAACYPHkyRo8eLdE9fvwY0dHRKCsrQ01NDVatWsWf\nd/q6BQsW4OHDh+A4DiUlJXj99dexdu1aSZuxsbFQq9Xo0KEDEhISmH0rLi5GbGwsLCws0LVrV0RH\nRzPP2VdffVXig9y5re8DS9e2bVusWLGC9yExMRFr1qwRaJydnbF06VKBB1qt1mCb+j6w2mzTpo3E\nB3d3d4muT58+Ah8SExORmpoq0GRnZ0s8WLVqFXM/Y2JiBD6w+ta6dWvExMQIfGDlBHNzc4kPcrmj\n1gdfX1+Jprq6WjIW7O3tJToAkrEAwGCb+j6w+lZdXS3wYeLEicjPzxdomjVrJhkLjo6OglixsbHY\nvHmzxIfVq1dL2tRoNBIfWH2rqqqS+AAIc66pqanBnMTKzaycpK97/vw5Myfpa2pqapj5yFCbrJwk\nbtNQTtLX2dnZGcxJ+roNGzYwc5K+5sWLFxIPWH2rrKxkeiC+ns2aNYvpg6HrntgHsS4gIECQk5KT\nkzF9+nSBJigoSJKTTExMDLYp9kGs8/f3Z/og1i1cuJDpg1j3/PlzPHjwAAB4H27cuCHZT5YP4lh+\nfn5MH8Rzhn/84x8SHwzNK8QeiHU9evSQeNCsWTOJrk+fPkwfXlY+SNsteP/xzIkGlH8CXANx5MgR\nLiIiguM4jjt//jw3e/Zspu7jjz/mPDw8uIkTJxqM9fXXX3OJiYkcx3HckydPODc3N6bu6NGj3JIl\nSziO47izZ88abLO6upoLDg7mRo4cyV2/ft1guy9evOC8vLwMbq9tZ9asWRzHcVxFRQW3YcMGWT3H\ncdzy5cu5L774grnt2LFj3Lx58ziO47j8/Hxu7ty5TN2uXbu4pUuXchzHcdevX+emTZvGbxMf01mz\nZnEFBQUcx3HcsmXLuKNHjzJ1jx494qZPn865u7tzmZmZBuP5+flxly9f5jiO4zIzM7mkpCSmbs6c\nOdz333/PcRzHRUREcEePHmX6XVxczAUGBgp+JtZ98cUX3Pbt2yXHQayLiIjgDh06xHEcx505c4Y7\nefIkU1fL77//zo0fP557+PChRBMcHMx9++23HMdxXFhYGHfixAlmrAkTJnDnz5/nOI7jUlJSuP37\n9wvO2d9//51zc3Nj+sA6tx8/fizxgRVP7MO0adMkmuDgYIkHhsaT2AeW7ssvv5T4wNKJfUhMTDQ4\nhvU9YO1nSEgIl5eXJ/CBpfP29pb4wMoJLB9YOvF4YGlYY4GlY/lgKF+JfWDpxOOBpWGNBbkcqe8D\nSxcSEiIZDywdazyIc66hnCTWGcpJ+rpr164xfRDHYuUjVpssD1g6QzlJrDOUkwxdh/R9EGsM5STx\n8WB5wLqesXxg6Vg+sHRiH+Lj4yUalg+GrrViH1g6lg8sHcsHuWt8rQ/37t2TaFg+sGKxfGDNGcQ+\npKamSjQsD1ixWGOBpWPlpJeZgE0ZgldD0mC/Tsg9DkQfZ2dnbNq0STbW6NGjERoaCkB3V1mtZq84\nGT58OFasWAFA95uinZ0dU7dq1SpMnjzZ6O35y5cv49mzZwgKCsLUqVNRVFQk0Zw6dQqdO3fGnDlz\nMHv2bAwZMkQ25oULF3D16lWDD9zu0KEDampqwHEcysrKYGZmxtRdvXoVgwYNAgB07NgR169f57eJ\nj2lxcTHeeOMNAMCgQYNw+vRppu7Zs2eYO3cuPD09BW2JdevXr0eXLl0A6O6QW1hYMHUbN26Ei4sL\nqqqq8ODBA9ja2ko0paWlSElJQVRUlGybxcXFOHnyJPz8/BAVFYVnz54xdT/88AN+++03vP/++8jO\nzsZbb73F1NXy0Ucfwc/PD82bN5dounXrhtLSUnAch4qKCv68E+vu37+P3r17AwBef/11FBYWCs7Z\nmpoamJqa4tKlSxIfWOc2ywdxPLVajZSUFIEPXbp0kWhYHrDafPLkicQHlq64uBgnTpwQ+CDWmZqa\nSnyYNWuWwTGs7wFrP7t164YnT54IfGAd399++03ig35OuHfvHuzs7Jg+sHJHZWWlwAeWhjUWWLoN\nGzZIfGDpWD6wdGIfBgwYINlP1liQy5H6PrB0rPEgPr5NmjRhjgf9nMtxHNMDQJqbxR7Uoq9TqVRM\nH8SxWGOB1SbLA5bOUE4S6wzlJEPXIX0fxJru3bszc5JYx/KAdT1j+cDSsXxg6cQ+lJWVSTQsH1ix\nWD6wdCwfWDqWD3LX+FofHjx4INGwfGDFYvkgnjO4ublJfDh69KhkXsHygDX/YI0Flo6Vk15mqqpr\nBK+GpMEmzoYeByLG3d0dpqamsrGsrKzwt7/9DeXl5QgNDcX8+fMNak1MTBAREYGEhASMHTtWsn3P\nnj1o3rw5Bg4cCM7IqhVLS0sEBQXhk08+QWxsLBYuXCjZh9LSUly8eBEfffQRYmNjERYWJhtz69at\nCAkJMbjd2toad+/exahRo7Bs2TL4+/szdd26dcPJkycB6Ba///vf/+b3R3xM9fdT/9ErYl27du3Q\nq1cvSVtiXYsWLQDoLgifffYZpk6dytSpVCrcu3cPY8eOxZMnT9C1a1eBpvbPvBEREbCyshL0Uxyr\nd+/eWLx4MXbt2gUnJyds2LCBqSspKYG9vT22b9+O1q1bY+vWrUwdADx+/Bhnz57FhAkTmJraP8GN\nGTMGjx8kJk+7AAAJyElEQVQ/xptvvsnUOTk54fvvvwcAnDhxApWVlcxzluUDS+fo6CjxgaVr3ry5\nwIcPPviAOU7EHohjhYaGIioqSuKDWDdv3jz06tUL4eHhAh9YfRP7kJ6ezuyb2ANWm7V/RtT3gdUm\nywfgPzkhPj4eHh4eBseDOHewfBBrDI0FsY41FsS6MWPGMH1gxevdu7fEB/F+GhoLrBwp9oGlY/kg\nPr5jx46V+HD58mVJztXPo7UesHIzywOWTuxDixYtJBqWB+JYNTU1TA/EOo7jmDmJpWP5YOg6pO8D\nKxbLA1Ys1lhgXc9YY4Gla9OmjcQHlq52yV6tDz4+Psw2xT6IYy1YsAARERESH1ht9ujRQ+IDS3f7\n9m2JD4au8fo+sDSOjo4SH1g6Jycn/hnBtT6I5wzieYW1tTXKy8sl8wrWWGDNP1g5iaUzlJNeVjTa\nGsGrIWmwLwfKPQ7kj/Drr78iJCQEfn5+eOedd2S1K1euxKNHj+Dr64uDBw/C0tKS31a7zjg/Px+X\nL19GeHg4Nm/ezE9C9OnQoQOcnZ35/9vb2+PBgwdo1aoVr7G3t0enTp2gVqvRsWNHWFhY4PHjx8wq\nNGVlZbh58yZ/wWGxY8cOuLq6Yv78+bh//z4CAgJw4MABmJubC3Te3t64du0apkyZgr59+6JHjx5Q\nqVTMmPrHXcmjV5Rw8OBBpKWlYevWrWjatKlBXdu2bZGTk4Mvv/wSSUlJWLlyJb+tuLgYt2/fRmxs\nLF68eIFr164hKSkJkZGRkjjDhw/nfxFzd3dHfHw8sz17e3v+rv/QoUORkpJisG+HDx+Gh4eHweOW\nkJCAzz77DJ06dUJGRgZWrlzJr5HUJzExEQkJCaipqYGLiwt/B17/nB0zZgxWr17Nf0bfB6XnNksn\n9oGlYXmgr2vfvr1BH8T7UFZWxvRBrEtKSpL4wOobywNxrAEDBjB9EOu6d+/O9AH4T07w8fHBixcv\nmD7o61i5w5AmNzeXORbEOkNjoVY3dOhQODg4GBwP+vEyMzP5u4v6PujvZ5MmTQyOBXHfDI0F8XEz\nNB70423evBmrV6/mfbh37x7y8/ORn5+PK1euIDw8HKWlpRIPlOZmQ7qzZ8/yPoSHh0s0qampEg/u\n3r0r0Hl6eqJdu3YSD4qLiwW6iIgIQd9qPbhw4QKvq91XU1NTiQ/nzp1j7kNOTg7vg/5+1sb66aef\nkJWVJfDg559/lvRt8eLF2LJli2AssK5nly5dkvig5LpXu42lKyws5H1wcHBAz549JRqxD3FxcYJY\n9+7dg6mpqcSHsLAwSZuDBg3i+1brA6tv9+7dk/hQ+z0Qcf+OHz/O+8CKFRERgezsbIEPEREREl1Y\nWBi2bNmC1NRU3gczMzPJnOH+/fsCH2xsbODq6mp0XmFo/nHmzBlBTjKkk7s+v2zUpXJgfdNgd5zl\nHgfCQu7u78OHDxEUFIRFixbBy8vLoC4rK4u/q2JhYQETExPJZH3Xrl1IT09Heno6unbtilWrVjEn\nzQDw9ddf8yfS/fv3UVFRAQcHB4HGxcUF3333Ha95/vy5wYlkQUEB+vXrZ7D/AGBnZ8d/ucDW1hYa\njYZ5p/7ChQvo378/MjIyMHLkSINfvAR0f96r/a3322+/hYuLi2C7sTvvYrKyspCRkYH09HQ4Ojoa\n1M2ePRu3bt0CoPstWt8LjuPQs2dPHDhwADt37sS6devw6quvMifNABAUFIQLFy4AAE6fPo0ePXow\ndS4uLvx5V1BQIPlyqP6+nj59ml/uwsLe3p73olWrVnj69ClTl5eXh7Vr12L79u148uQJBgwYwDxn\nu3XrJvFB6bnN0ol9YGlYHoh1vXr1YvrAisfygaUT+9C2bVvmfoo9YMVi+cDSsXxg5YTXXnsN586d\nE/igJHewNDk5OZKxINapVCoEBwdLfBDrHBwccPDgQYkPrHhz587Fjz/+yB9Dc3NzSd/eeOMN/q9S\ntWPB0H6KfWDp7O3tYW1tLfCBpTt58qTAh/j4eEHOTU5Ohqurq2QsKM3NLN2pU6cEPuhrunXrhlWr\nVmHp0qUSD8SxDh48iCNHjkg8YMWbPXu2wIMePXpI4iUnJ8PNzU3iA0vXvHlzgQ8sTfv27SVjgXU8\nLl68KBkL4utZeXk5Bg4cKBkLSq57APv6ePbsWYEPrDaXLVsm8UGs69ixI3JyciQ+sOLNmTNH4gNL\nN3z4cIkPhvZV3wdWrPbt20vGAitWUVGRxAfxnKGyshL9+vUT+DBgwABF8wrW/OPbb7+V5CRWm1FR\nUQavzy8jdXkcXX3TYHecWY8DkcPQXT8ASEtLw9OnT5GamopNmzZBpVJh27ZtkruwI0aMQGRkJPz8\n/KDRaBAVFSXRKG0TAHx8fBAZGYn33nsPJiYmSExMlJxcbm5u+P777+Hj4wOO4xATE2Mw7o0bN2Qn\nuAAQGBiIJUuWYMqUKdBoNAgLC2Pe9XJ2dsaHH36ILVu2oEmTJoJvV4sJDw/H0qVLUV1djU6dOmHU\nqFGC7caOgz5arRaJiYlo27YtgoODoVKp8OabbzKXn8yYMQMREREwNzeHlZWV4C5xXdoEgNjYWKxY\nsQJmZmZwcHBAXFwcUxceHo7o6Gh8/vnnsLW1xdq1awXb9du9efOmrB8rVqzAvHnzoFarYW5uzq/l\nFOPs7IzAwEBYWVnhrbfewqBBg5CQkCA5Z6OiohAfHy/wITExUdG5LR4DWq0WV69eFfjw/PlzSaz5\n8+dLPFA6nli6yMhIJCYmCnxISUmR6FatWoWoqCjeh7Zt20o0H3/8scQDVpssH1i6adOmSXyorKwU\n5ITo6Gi88soriI6OFvjw/Plzo7lDnF+WLFmCyMhIyVgICgqSxGrWrJnEB2tra0X5ipXX2rRpg7i4\nON6HhIQExMfHC/aza9euiIqKQmZmJj8W1Go1s02xD6w27e3tMX/+fIEPTZs2lRwTlUol8UFMfeUk\nlUrFP7XFWE6aOXOmwXxUG6suNxGWL18u8MBYTtL3wVC7xnJSfHy8bE6qPW6snFRdXS24nq1cuZJ/\nAoy+DxqNxuh1D5BeHxMSEvin4NT64OLigrKyMl5TOw8Q+2BnZydpk3UOiNtMSkqChYWFxAdzc3PJ\nvjo4OEh8sLS0lMQzMTER+MBqU6vVSsaCg4ODZB9KS0uZY0F/zhAbG8s/7abWh7CwMKxdu9bovEI8\n/1i2bBkWLFjAHAvieYq1tbXseHjZMFZm+8+kwR5HRxAEQRAEQRCNmZf7XjxBEARBEARBvCTQxJkg\nCIIgCIIgFEATZ4IgCIIgCIJQAE2cCYIgCIIgCEIBNHEmCIIgCIIgCAXQxJkgCIIgCIIgFEATZ4Ig\nCIIgCIJQwP8ApBJTe8CwXZAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b67574518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAABUCAYAAACSjrEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXlYFMfa9u+BYTGA4EJcENGY43qiRnISo0FxwWhEIgIa\nlMWIcQODigoICiqbuBEXFOOJRiRiFhU1KmKIeORzQXJJFKOJe8TE1wUVEIFh+v1jDn2mu2t6mryE\nyPme33XNpUPd81R1313VRVHTj4rjOA4EQRAEQRAEQchi8lc3gCAIgiAIgiCaAjRxJgiCIAiCIAgF\n0MSZIAiCIAiCIBRAE2eCIAiCIAiCUABNnAmCIAiCIAhCAeq/ugEEQRAEQRAEoZRf3nlX8P5vJ7Mb\nrW6aOBMEQRAEQRBNBxPTv6xqmjgTBEEQBEEQTQaV+q+bvtIeZ4IgXji6d+8ODw8PjB07Fp6envy/\nd+/eNfiZixcvIjQ0tMHakJ2dDX9//3p9JicnBx4eHvD09ERgYCB+/fVXvuz48ePw8PDAqFGjMGfO\nHFRUVCgqq6OkpAQzZszA66+/jkOHDgEA9u3bh9deew0pKSl4/PgxACAxMREBAQH48ccf/8hhEwRB\nvPCozM0Er0atmzIHEgTxotGjRw+cPn0atra2f1kbsrOzkZGRgR07dijSV1VVoX///ti/fz8cHR2x\nfft2nDp1CmlpaXj06BHc3d2xe/duODo6YtWqVaioqEBMTIxsmZh9+/Zh/fr1+O677wAAFRUVGDhw\nIL799ls4ODgAAI4cOYJhw4bBzKxxbyYEQRCNxXUPX8H7V/bvarS6acWZIIgXDo7jYOh3+rNnz8Lb\n2xvBwcHw8PDAhAkTcO3aNZw9exZjxowBADx79gyhoaHw9PTEuHHjsGTJEv7zu3fvxpgxYzB27FgE\nBQXh5s2bfNknn3wCNzc3jB8/Hjk5OfzPc3NzMX78eIwbNw4TJ07E+fPnJe2qra0FAJSVlfFtsLCw\nAADk5+ejd+/ecHR0BAD4+vriwIEDRsvEtGjRQvD+wIEDaNOmDUpLSwEAd+7cQZs2bWjSTBDEfzUq\ntVrwakxojzNBEC8kAQEBMDXVfQGE4zg4Ojpi/fr1AICffvoJixYtQr9+/ZCZmYmFCxciPDyc/2xO\nTg6ePXuGvXv3QqvVIjY2Fr/++itKSkrw2WefYffu3bCzs8PevXsRHByMb7/9FseOHcOxY8ewf/9+\nWFhYYNasWQCAW7duYe3atdi5cydsbW1x9epVTJ48GceOHYOlpSVf50svvYSYmBhMmDABLVq0gFar\nxa5dulWQ3377DW3btuW1bdu2RUVFBSoqKmTLrKysBOfEzs6O//8PP/yAbt26oVWrVvzE+YcffoCH\nh0eDnH+CIIgXlcbenqEPTZwJgnghSU9PN7hVo1u3bujXrx8AwMvLC8uWLcOTJ0/4cmdnZ6SkpMDf\n3x8DBw5EYGAgHB0dkZmZiVGjRvETUE9PTyQkJKCkpASnT5+Gm5sbmjVrxsdNT09Hfn4+7t+/j8mT\nJ/Or4Gq1Grdu3UK3bt34On/++Wekpqbi8OHD6NChA9LT0xESEoKsrCyDq+empqayZWLqVpyrq6tR\nWFiIjz76CHZ2digtLcWpU6fQv39/ALpV71OnTuHGjRuYPn264ZNMEATRBFGZmyvWajQaLFq0CCUl\nJaipqcGMGTMwdOhQvjw3NxepqalQq9Xw8vKCj4+PbDyaOBME8UIi9/ULtd6f5up0Jib/2XnWoUMH\nHD16FGfPnsXp06cRGBiIxYsXQ6vVSmJptVpoNBpJnfqr3QMGDMCaNWv4st9//x1t2rQRxDl58iSc\nnZ3RoUMHAMCkSZOQmJiIx48fo127digqKhJ8vnnz5rC0tJQtE9OiRQtwHIeMjAx4e3sD0K1C3717\nF5aWlnj55ZcBADY2NujVqxd+/vlng+eQIAiiqVKf7Rn79+9HixYtkJycjCdPnmDs2LH8xFmj0SAp\nKQl79uyBhYUFfH19MWzYMLRs2dJgPNrjTBBEk+PSpUv8pHD37t3o168fbGxs+PJdu3YhIiICAwcO\nRFhYGFxcXPDLL7/AxcUFhw8fxqNHjwAA33zzDVq0aAEnJye4uLjgyJEjKCsrg1arRVZWFgCgf//+\nyM/Px/Xr1wEAeXl5eP/991FVVSVoU8+ePXH27Fk8fPgQgG67iKOjI+zs7PDOO+/gxx9/xO3bt/k2\nDxs2DABky8TY2NigrKwMzZs351efW7RogZMnT2L48OH/9xNLEATRBKjPUzVGjRrFP3FJq9UKFl6u\nXbsGJycnWFtbw8zMDM7OzigoKJCNRyvOBEG8cKhUKskeZ5VKhblz58LS0hL29vZYu3Yt7ty5g9at\nWyM5ORl37tzhPz927FgUFBTgvffeQ7NmzeDg4IDAwEDY2NggMDAQgYGBAHSTzrS0NADA4MGD8csv\nv8DLywu2trbo3r07SktL0aVLFyxbtgzz5s0DoFuJ3rRpk2RFuH///ggKCoK/vz/Mzc1ha2uL1NRU\nAEDLli2RkJCA2bNnQ6PRwNHREcnJyUbLWPTv3x9eXl78e3t7e8yZM0ew4k4QBPHfTH22atRtvysv\nL0doaCjmzp3Ll5WXlwsWXaysrPgveBusmx5HRxBEU+Ls2bNYvny5wSdPEDpKSkqwd+9ehISE/NVN\nIQiCaFBK5i0SvHdYkyCr/+233xASEgI/Pz94enryP79y5QpWr16NLVu2ANA9B9/Z2RkjRowwGIuW\nKAiCIP7LqKioQHZ2Ni5evIhffvnlr24OQRBEg6IyNxe85Hjw4AGCgoKwYMECwaQZALp06YJbt27h\n6dOnqK6uRkFBAfr27StfN604EwRBEARBEE2F3xbHC963Wx5lUBsfH4/Dhw/jlVde4bf9jR8/HpWV\nlfDx8cHx48exYcMGcBwHb29v+Pr6GowF0MSZIAiCIAiCaEL8vmyF4H3bJeEGlA0PfTmQIAiCIAiC\naDLU58uBDc2fPnFef+SkbPnske8AANZ+myermzt6MJIPfG+0voVjhmDj0XyjuuARA/Fhqnxu822z\ndMv1H6TskNVlzgkAAASmfiGr+3zWRPis3W60bV/NnQyP5K2ymv0LpwIAlu/JkdUtHucGAIj9OltW\nF+v9LtKOnTLatunD38aqg8dlNfPdXQEAE1I+l9XtnqN7skHAxgxZ3Y7gSRi3epvRtu0J+xBbc8/I\naqYOfQsAELnrW1ldou9oAMDMrV/L6jZN9casf8prACA1yFvxMXivkdd9Pe9DAMC8HVmyujUB7wMA\nQrfvldV9MtlTcd+atuVLWc2WaePr1TYlfcuYB4DOh0nr0mU1GR/7AwA+3rZHVrfuw3EAlHm/5bvT\nRts2bVh/zPj0K1nN5o90D9w3di2lBume3RyVeUhWF//Be1j2zVGjbVviNQILM4x/yTJ50hij18jC\nMUMAwOg5mTZMl6RFyfiQuO87o22LHDsMKYdOyGrmvDcIABBnZLyM/vd4GfKZ/DWyYco4o5o6ndLz\npvT6VTL+jl35T6Nt27cgSPG9LWHvMVndIk/doxCVjF1Kz5uxsQbQjTeeqz6T1eydPwUAjF7nyZPG\nAFDmw4Kd+422baWfB1Yb8Srs3/dKJfdnQNn129Dj5YuKyowyBxIEQRAEQRCEUeqTAKWhoYkzQRAE\nQRAE0WRQWfx1WzUUP46OlaqWIAiCIAiCIBoTlZmZ4KWEoqIi+PtLt6Bs374d7u7uCAgIQEBAAG7e\nvCkbR3bF+ddff0ViYiIuXrwItVoNrVaLrl27IjIyEp07d1bUUIIgCIIgCIJoKOq7VWPr1q3IysqC\nlZWVpKy4uBjJycno2bOnoliyNUdFRSEsLAx9+vThf3b+/HlERkYiMzOzXo0mCIIgCIIgiP8r9X2q\nhpOTEzZu3IiFCxdKyoqLi5GWlob79+/D1dUV06ZNk40lu1WjurpaMGkGYDSjCkEQBEEQBEH8WajM\nzQQvY7i5ucHU1JRZNnr0aCxduhQ7duxAYWEh8vLkn/Imu+LcrVs3REZGwsXFBTY2NqioqEBeXh66\ndetmtJEEQRAEQRAE0dCozBru2RaBgYGwtrYGAAwePBiXLl3C4MGDDepla46NjcWxY8dQWFiI8vJy\nWFtbY8iQIXBzc2uwBhMEQRAEQRCEUv7oc5zFybLLy8vh7u6Ow4cPw9LSEqdPn4a3t7d83ZRymyAI\ngiAIgmgqlH0n3E5hM8zwCnEdJSUlCAsLQ2ZmJg4ePIjKykr4+Phg//792LFjBywsLPD2228jJCRE\nNs6fPnFWkoEJAPzW75TV7Zzth08O/8tofaGjXIzGqovnFrdJVpMTPROAskxNgLKMZMYyJAK6LIkN\nmXkNgKLsVcZi1cVT2jal523qZvkvmm6d8UGDZ4NUknkNAMLS5TNErfb3QMxXR4y2banPSMWZsJRk\n+gOAz0+ck9UFDnoDgLJsXkozBy7+8rCsZvn4UQCUZ+dTksHRWHZMQJch86O03bKaT6dPAACjWdX2\nLQgCAPh+Ip9Za1eov+LMdkquIwCYskm+L3w28wNdTAXnTek4qDTD4MiENFnNkUXTASjPZjr3832y\nurWBY40eJ6A7VqXZ+ZRmZjR2TpZ4jTCaxQ3QZXJTOl4qzWyn5PrNyP/BaNsmDeynOEOt0myQxu7R\noaNcjGaMBHRZI5VmtHSN3SCrOR6rmwQpHZOUzEWUtk1pplVjGYW/mjsZgLL7lrH7KaC7pyoZ315k\nyvOEcwLrwQMbrW5KgEIQBEEQBEE0GZR8IfDPgibOBEEQBEEQRJPhj+5xbgho4kwQBEEQBEE0GWji\nTBAEQRAEQRAKoK0aBEEQBEEQBKGA+mYObEhkJ87+/v6oqakR/IzjOKhUKkq5TRAEQRAEQTQ6f2Sr\nRlFREVatWoX0dOETRXJzc5Gamgq1Wg0vLy/4+PjIxpGdOM+fPx/R0dHYuHGjwVSFBEEQBEEQBNFY\nqNT12zCxdetWZGVlwcrKSvBzjUaDpKQk7NmzBxYWFvD19cWwYcPQsmVLg7FM5Crq06cP3n//fVy5\ncgUODg6CF0EQBEEQBEE0Ns9trAQvYzg5OWHjxo2Sn1+7dg1OTk6wtraGmZkZnJ2dUVBQIBuLMgcS\nBEEQBEEQTYaysjLBexsbG6Of0c8cWEdhYSEyMjKwZs0aAMC6devQvn172bTbf/qXA1cbyR4X9u/s\ncUoyNaUcOmG0vjnvDcKG7JNGdSHvvgPPVZ/JavbOnwJAWQYxQFl2vplb5TNXAcCmqd6KM1wZy1wW\nOXYYAGDcavnscXvCPlScAU9JJjpAeXY+Yxm4ose5Kc7SpTSDmLEMjnNH69J3hn9xUFa3YqI7onfL\nZ9MDgLgJo4xm3QN0mfeUZrhSmm1KSbY/pdnjlHgFAAEbM2R1O4InAYCiLInGriNAdy01ZGYwpTql\nWdCUZtNTMg4CwPoj8mPc7JHvGD23gO78BqZ+YVT3+ayJivuz0rFLSQY8pWO+0gx4xu4NIe++A0DZ\n9WssKy6gy4xr7Px+PmsiAOWZA5Vcl0ozWirNUmrsvrVpqs5TJZlbjV0fgO4aMZatENBlLFSSrRBQ\n3reU3N+UzjGUnt+GHFcX7JTPUgoAK/08sCnn/8lqZroNMBrnvwFra2uUl5fz7ysqKtC8eXPZz8hu\n1SAIgiAIgiCIF4lqEzPBSyniTRZdunTBrVu38PTpU1RXV6OgoAB9+/aVjUGPoyMIgiAIgiCaDNUa\nzR/6nEqlAgAcPHgQlZWV8PHxQWRkJKZMmQKO4+Dj44OXX35ZNka9J87V1dUw/wufn0cQBEEQBEH8\n/4tGq633ZxwcHPj9ze7u7vzPXV1d4erqqjiOwa0aubm5GDJkCNzc3HDo0H/2tk2dOrXejSUIgiAI\ngiCIhqC6plbwakwMrjhv3rwZ+/btg1arRWhoKKqqquDp6SnZH0IQBEEQBEEQjUVN7R/bqtEQGJw4\nm5mZwdbWFgCQmpqKwMBAtGvXjt8fQhAEQRAEQRCNjaa2/ls1GgqDWzUcHByQmJiIZ8+ewdraGhs2\nbMCyZctw/fr1xmwfQRAEQRAEQfBUaTSCV2NiMAGKRqPB/v37MWrUKDRr1gwA8ODBA6SlpSEqKqpR\nG0kQBEEQBEEQAHD66m3B+/6vdjSo5TgOsbGxuHLlCszNzREfHw9HR0e+fPv27fj666/5NNvLli1D\np06dDMajzIEEQRAEQRBEk+HklZuC9+9062RQm5OTg9zcXCQmJqKoqAhpaWlITU3lyxcsWIAPP/wQ\nPXv2VFT3n/4cZ2NZ1eImjAIARdmVPjt+1mh9U1zfNJqlC9Bl6lKacSgsXT4Tz2p/DwDKsvMZq7Ou\nXqUZF5VmaBuZkCarO7JouuIMYkqzHM39fJ+sbm3gWADKMmEpzSBmLPvhUp+RAJRlLQOAVUZ8mO/u\nishd3xptW6LvaGw8mm9UFzxioNF4ib6jAQCT1slntMz4WJfRcu+5i7I6zzf+bjQLIaDLRKgkYx0A\nvJe4RVZ3KHIaAGXnd8X+XKNtC/cYio/SdstqPp0+AYDyjItK+pbSjJYJe4/JahZ5DgcAo8ca7jEU\nADA6Sf78fhsxDfN2ZBlt25qA9zFk6Uajuu9jgvHl6SJZzfj+fQAoGwcBZdeIsfMG6M6d0ux8Ssdy\nJePv1M2ZshoA2DrjA8X3GSUZPgEoypCp1NPdp87Laia8rUsEYWz8nfPeIADK+paxbKyALiOr0mym\nxuYFU1zfBKB8LqLkGlGalVPp+f38xDlZXeCgNwAoy2KsNNPqzpOFshq/d5yNxvkrqc9znAsLC+Hi\noutnffr0wcWLwnticXEx0tLScP/+fbi6umLatGmy8SgBCkEQBEEQBNFkqK5V/gi68vJy2NjY8O/V\najW0Wi1MTHRf8xs9ejQmTZoEa2trBAcHIy8vD4MHDzYYj1JuEwRBEARBEE2GGo1W8JLD2toaFRUV\n/Hv9STMABAYGws7ODmq1GoMHD8alS5dk49Vr4vz8+XNUV1fX5yMEQRAEQRAE0WDU1GoELzn69euH\nvLw8AMD58+fRtWtXvqy8vBzu7u6orKwEx3E4ffo0evXqJRtPdqvG1atXsWbNGtja2mLMmDGIjo6G\niYkJoqKiMGTIEKXHRxAEQRAEQRANQo1G+VYNNzc35Ofn44MPPgAAJCYm4uDBg6isrISPjw/mzZsH\nf39/WFhY4O2338agQYNk48lOnGNiYhAaGoqSkhJ8/PHHyM7OhoWFBaZOnUoTZ4IgCIIgCKLRqanH\nHmeVSoWlS5cKfta5c2f+/x4eHvDw8FAcT3birNVq8eabum+knjlzBq1atdJ9SE3fKSQIgiAIgiAa\nn+p6rDg3NLJ7nDt37oyoqChotVokJSUBALZs2YLWrVs3SuMIgiAIgiAIQp8aTa3g1ZjIJkDRarXI\nzc3F8OHD+Z9lZWVhxIgRfDZBgiAIgiAIgmgsNmQLcwqEvPtOo9Utu+fCxMREMGkGgPfff/9PbRBB\nEARBEARBGKKxV5n1oc3KBEEQBEEQRJOhPglQOI5DbGwsrly5AnNzc8THx8PR0ZEvz83NRWpqKtRq\nNby8vODj4yMbjybOBEEQBEEQRJOhPivOx44dQ3V1NTIzM1FUVITExESkpqYCADQaDZKSkrBnzx5Y\nWFjA19cXw4YNQ8uWLQ3Go8yBBEEQBEEQRJOhprZW8JKjsLAQLi4uAIA+ffrg4sWLfNm1a9fg5OQE\na2trmJmZwdnZGQUFBbLxaMWZIAiCIAiCaDLU53F05eXlsLGx4d+r1Wo+7ba4zMrKCmVlZbLxaOJM\nEARBEARBNBk09djjbG1tjYqKCv593aS5rqy8vJwvq6ioQPPmzWXj0VYNgiAIgiAIoslQrakVvOTo\n168f8vLyAADnz59H165d+bIuXbrg1q1bePr0Kaqrq1FQUIC+ffvKxqMVZ4IgCIIgCKLJUJ+tGm5u\nbsjPz8cHH3wAAEhMTMTBgwdRWVkJHx8fREZGYsqUKeA4Dj4+Pnj55Zdl48kmQGlIjD0ORJ+ioiKs\nWrUK6enpzHKNRoNFixahpKQENTU1mDFjBoYOHSrRabVaREdH48aNGzAxMcHSpUvx6quvMmM+fPgQ\nXl5e2LZtmyCHuZhx48bB2toaANChQwckJCRINFu2bEFubi5qamowceJEeHl5STR79+7Fnj17oFKp\nUFVVhcuXLyM/P5+PrX+s4eHhKCkpgVqtxvLly5ntq66uRmRkJO7cuQNra2vExMSgY8eOfLn+Ob19\n+zYiIiJgYmKCv/3tb4iJiWHq6khMTMQrr7yCCRMmMHU//fQT4uLiYGpqCnNzcyQnJ/PfSNXXXb16\nFUuWLAEAODk5IT4+HiYmJsw6Dxw4gIyMDGRmZhqsc/r06ejUqRMAwNfXF6NGjZLoHj16hOjoaJSV\nlaG2thYrVqzgrzt93bx58/DgwQNwHIeSkhK8/vrrWL16taTO2NhYqNVqdOrUCfHx8cy2FRcXIzY2\nFhYWFujevTuio6OZ1+yrr74q8UHu2tb3gaVr3749li9fzvuQkJCAVatWCTROTk5YvHixwAOtVmuw\nTn0fWHW2a9dO4oObm5tE17dvX4EPCQkJSE1NFWgOHjwo8WDFihXM44yJiRH4wGpb27ZtERMTI/CB\nNSaYm5tLfJAbO+p88PHxkWhqamokfcHOzk6iAyDpCwAM1qnvA6ttNTU1Ah8mTJiA/Px8gaZly5aS\nvuDg4CCIFRsbi02bNkl8WLlypaROjUYj8YHVturqaokPgHDMNTU1NTgmscZm1pikr3v+/DlzTNLX\n1NbWMscjQ3WyxiRxnYbGJH2dra2twTFJX7d+/XrmmKSvqaqqknjAaltlZSXTA/H9bMaMGUwfDN33\nxD6IdQEBAYIxKTk5GVOnThVogoKCJGOSiYmJwTrFPoh1/v7+TB/Euvnz5zN9EOueP3+O+/fvAwDv\nw40bNyTHyfJBHMvPz4/pg3jO8I9//EPig6F5hdgDsa5Xr14SD1q2bCnR9e3bl+nDi8pHabsF7z+d\nPsGA8k+AaySOHj3KRUREcBzHcefPn+dmzpzJ1H366aecu7s7N2HCBIOxvvnmGy4hIYHjOI57/Pgx\n5+rqytTl5ORwixYt4jiO486cOWOwzpqaGi44OJh79913uevXrxust6qqivP09DRYXlfPjBkzOI7j\nuIqKCm79+vWyeo7juKVLl3Jffvkls+zYsWPcnDlzOI7juPz8fG727NlM3c6dO7nFixdzHMdx169f\n56ZMmcKXic/pjBkzuIKCAo7jOG7JkiVcTk4OU/fw4UNu6tSpnJubG5eZmWkwnp+fH3f58mWO4zgu\nMzOTS0xMZOpmzZrFnTt3juM4jouIiOBycnKYfhcXF3OBgYGCn4l1X375Jbdt2zbJeRDrIiIiuMOH\nD3Mcx3GnT5/mjh8/ztTV8eTJE27s2LHcgwcPJJrg4GDuxIkTHMdxXFhYGPf9998zY40bN447f/48\nx3Ecl5KSwu3fv19wzT558oRzdXVl+sC6th89eiTxgRVP7MOUKVMkmuDgYIkHhvqT2AeW7quvvpL4\nwNKJfUhISDDYh/U9YB1nSEgIl5eXJ/CBpfPy8pL4wBoTWD6wdOL+wNKw+gJLx/LB0Hgl9oGlE/cH\nlobVF+TGSH0fWLqQkBBJf2DpWP1BPOYaGpPEOkNjkr7u2rVrTB/EsVjjEatOlgcsnaExSawzNCYZ\nug/p+yDWGBqTxOeD5QHrfsbygaVj+cDSiX2Ii4uTaFg+GLrXin1g6Vg+sHQsH+Tu8XU+3L17V6Jh\n+cCKxfKBNWcQ+5CamirRsDxgxWL1BZaONSa9yARszBC8GpNG+3VC7nEg+jg5OWHjxo2ysUaNGoXQ\n0FAAulVltZq942T48OFYvnw5AN1vira2tkzdihUr4Ovra3R5/vLly3j27BmCgoIwefJkFBUVSTQn\nT55E165dMWvWLMycORNDhgyRjXnhwgVcvXrV4AO3O3XqhNraWnAch7KyMpiZmTF1V69exaBBgwAA\nnTt3xvXr1/ky8TktLi7GG2+8AQAYNGgQTp06xdQ9e/YMs2fPhoeHh6AusW7t2rXo1q0bAN0KuYWF\nBVO3YcMGODs7o7q6Gvfv34eNjY1EU1paipSUFERFRcnWWVxcjOPHj8PPzw9RUVF49uwZU/fDDz/g\n999/x4cffoiDBw/irbfeYurqWLduHfz8/NCqVSuJpkePHigtLQXHcaioqOCvO7Hu3r176NOnDwDg\n9ddfR2FhoeCara2thampKS5duiTxgXVts3wQx1Or1UhJSRH40K1bN4mG5QGrzsePH0t8YOmKi4vx\n/fffC3wQ60xNTSU+zJgxw2Af1veAdZw9evTA48ePBT6wzu/vv/8u8UF/TLh79y5sbW2ZPrDGjsrK\nSoEPLA2rL7B069evl/jA0rF8YOnEPgwYMEBynKy+IDdG6vvA0rH6g/j8Nm/enNkf9MdcjuOYHgDS\nsVnsQR36OpVKxfRBHIvVF1h1sjxg6QyNSWKdoTHJ0H1I3wexpmfPnswxSaxjecC6n7F8YOlYPrB0\nYh/KysokGpYPrFgsH1g6lg8sHcsHuXt8nQ/379+XaFg+sGKxfBDPGVxdXSU+5OTkSOYVLA9Y8w9W\nX2DpWGPSi0x1Ta3g1Zg02sTZ0ONAxLi5ucHU1FQ2VrNmzfDSSy+hvLwcoaGhmDt3rkGtiYkJIiIi\nEB8fjzFjxkjK9+zZg1atWmHgwIHgjOxasbS0RFBQEP75z38iNjYW8+fPlxxDaWkpLl68iHXr1iE2\nNhZhYWGyMbds2YKQkBCD5VZWVrhz5w5GjhyJJUuWwN/fn6nr0aMHjh8/DkC3+f1//ud/+OMRn1P9\n49R/9IpY16FDB/Tu3VtSl1jXunVrALobwhdffIHJkyczdSqVCnfv3sWYMWPw+PFjdO/eXaCp+zNv\nREQEmjVrJminOFafPn2wcOFC7Ny5E46Ojli/fj1TV1JSAjs7O2zbtg1t27bFli1bmDoAePToEc6c\nOYNx48YxNXV/ghs9ejQePXqEN998k6lzdHTEuXPnAADff/89KisrmdcsyweWzsHBQeIDS9eqVSuB\nDx999BFK2rGYAAAJnElEQVSzn4g9EMcKDQ1FVFSUxAexbs6cOejduzfCw8MFPrDaJvYhPT2d2Tax\nB6w66/6MqO8Dq06WD8B/xoS4uDi4u7sb7A/isYPlg1hjqC+Iday+INaNHj2a6QMrXp8+fSQ+iI/T\nUF9gjZFiH1g6lg/i8ztmzBiJD5cvX5aMufrjaJ0HrLGZ5QFLJ/ahdevWEg3LA3Gs2tpapgdiHcdx\nzDGJpWP5YOg+pO8DKxbLA1YsVl9g3c9YfYGla9euncQHlq5uy16dD97e3sw6xT6IY82bNw8RERES\nH1h19urVS+IDS3f79m2JD4bu8fo+sDQODg4SH1g6R0dH/hnBdT6I5wzieYWVlRXKy8sl8wpWX2DN\nP1hjEktnaEx6UdFoawWvxqTRvhwo9ziQP8Jvv/2GkJAQ+Pn54b333pPVJiUl4eHDh/Dx8cGhQ4dg\naWnJl9XtM87Pz8fly5cRHh6OTZs28ZMQfTp16gQnJyf+/3Z2drh//z7atGnDa+zs7NClSxeo1Wp0\n7twZFhYWePToETMLTVlZGW7evMnfcFhs374dLi4umDt3Lu7du4eAgAAcOHAA5ubmAp2XlxeuXbuG\nSZMmoV+/fujVqxdUKhUzpv55V/LoFSUcOnQIaWlp2LJlC1q0aGFQ1759e2RnZ+Orr75CYmIikpKS\n+LLi4mLcvn0bsbGxqKqqwrVr15CYmIjIyEhJnOHDh/O/iLm5uSEuLo5Zn52dHb/qP3ToUKSkpBhs\n25EjR+Du7m7wvMXHx+OLL75Aly5dkJGRgaSkJH6PpD4JCQmIj49HbW0tnJ2d+RV4/Wt29OjRWLly\nJf8ZfR+UXtssndgHloblgb6uY8eOBn0QH0NZWRnTB7EuMTFR4gOrbSwPxLEGDBjA9EGs69mzJ9MH\n4D9jgre3N6qqqpg+6OtYY4chTW5uLrMviHWG+kKdbujQobC3tzfYH/TjZWZm8quL+j7oH2fz5s0N\n9gVx2wz1BfF5M9Qf9ONt2rQJK1eu5H24e/cu8vPzkZ+fjytXriA8PBylpaUSD5SOzYZ0Z86c4X0I\nDw+XaFJTUyUe3LlzR6Dz8PBAhw4dJB4UFxcLdBEREYK21Xlw4cIFXld3rKamphIfzp49yzyG7Oxs\n3gf946yL9dNPPyErK0vgwc8//yxp28KFC7F582ZBX2Ddzy5duiTxQcl9r66MpSssLOR9sLe3x2uv\nvSbRiH1YtmyZINbdu3dhamoq8SEsLExS56BBg/i21fnAatvdu3clPtR9D0Tcvu+++473gRUrIiIC\nBw8eFPgQEREh0YWFhWHz5s1ITU3lfTAzM5PMGe7duyfwwdraGi4uLkbnFYbmH6dPnxaMSYZ0cvfn\nF436ZA5saBptxVnucSAs5FZ/Hzx4gKCgICxYsACenp4GdVlZWfyqioWFBUxMTCST9Z07dyI9PR3p\n6eno3r07VqxYwZw0A8A333zDX0j37t1DRUUF7O3tBRpnZ2f861//4jXPnz83OJEsKChA//79DbYf\nAGxtbfkvF9jY2ECj0TBX6i9cuIC3334bGRkZePfddw1+8RLQ/Xmv7rfeEydOwNnZWVBubOVdTFZW\nFjIyMpCeng4HBweDupkzZ+LWrVsAdL9F63vBcRxee+01HDhwADt27MCaNWvw6quvMifNABAUFIQL\nFy4AAE6dOoVevXoxdc7Ozvx1V1BQIPlyqP6xnjp1it/uwsLOzo73ok2bNnj69ClTl5eXh9WrV2Pb\ntm14/PgxBgwYwLxme/ToIfFB6bXN0ol9YGlYHoh1vXv3ZvrAisfygaUT+9C+fXvmcYo9YMVi+cDS\nsXxgjQl///vfcfbsWYEPSsYOliY7O1vSF8Q6lUqF4OBgiQ9inb29PQ4dOiTxgRVv9uzZ+PHHH/lz\naG5uLmnbG2+8wf9Vqq4vGDpOsQ8snZ2dHaysrAQ+sHTHjx8X+BAXFycYc5OTk+Hi4iLpC0rHZpbu\n5MmTAh/0NT169MCKFSuwePFiiQfiWIcOHcLRo0clHrDizZw5U+BBr169JPGSk5Ph6uoq8YGla9Wq\nlcAHlqZjx46SvsA6HxcvXpT0BfH9rLy8HAMHDpT0BSX3PYB9fzxz5ozAB1adS5Yskfgg1nXu3BnZ\n2dkSH1jxZs2aJfGBpRs+fLjEB0PHqu8DK1bHjh0lfYEVq6ioSOKDeM5QWVmJ/v37C3wYMGCAonkF\na/5x4sQJyZjEqjMqKsrg/flFpD6Po2toGm3FmfU4EDkMrfoBQFpaGp4+fYrU1FRs3LgRKpUKW7du\nlazCjhgxApGRkfDz84NGo0FUVJREo7ROAPD29kZkZCQmTpwIExMTJCQkSC4uV1dXnDt3Dt7e3uA4\nDjExMQbj3rhxQ3aCCwCBgYFYtGgRJk2aBI1Gg7CwMOaql5OTEz755BNs3rwZzZs3F3y7Wkx4eDgW\nL16MmpoadOnSBSNHjhSUGzsP+mi1WiQkJKB9+/YIDg6GSqXCm2++ydx+Mm3aNERERMDc3BzNmjUT\nrBLXp04AiI2NxfLly2FmZgZ7e3ssW7aMqQsPD0d0dDR27doFGxsbrF69WlCuX+/Nmzdl/Vi+fDnm\nzJkDtVoNc3Nzfi+nGCcnJwQGBqJZs2Z46623MGjQIMTHx0uu2aioKMTFxQl8SEhIUHRti/uAVqvF\n1atXBT48f/5cEmvu3LkSD5T2J5YuMjISCQkJAh9SUlIkuhUrViAqKor3oX379hLNp59+KvGAVSfL\nB5ZuypQpEh8qKysFY0J0dDReeeUVREdHC3x4/vy50bFDPL4sWrQIkZGRkr4QFBQkidWyZUuJD1ZW\nVorGK9a41q5dOyxbtoz3IT4+HnFxcYLj7N69O6KiopCZmcn3BbVazaxT7AOrTjs7O8ydO1fgQ4sW\nLSTnRKVSSXwQ01Bjkkql4p/aYmxMmj59usHxqC5WfRYRli5dKvDA2Jik74Oheo2NSXFxcbJjUt15\nY41JNTU1gvtZUlIS/wQYfR80Go3R+x4gvT/Gx8fzT8Gp88HZ2RllZWW8pm4eIPbB1tZWUifrGhDX\nmZiYCAsLC4kP5ubmkmO1t7eX+GBpaSmJZ2JiIvCBVadWq5X0BXt7e8kxlJaWMvuC/pwhNjaWf9pN\nnQ9hYWFYvXq10XmFeP6xZMkSzJs3j9kXxPMUKysr2f7womEszfafSaM9jo4gCIIgCIIgmjIv9lo8\nQRAEQRAEQbwg0MSZIAiCIAiCIBRAE2eCIAiCIAiCUABNnAmCIAiCIAhCATRxJgiCIAiCIAgF0MSZ\nIAiCIAiCIBRAE2eCIAiCIAiCUMD/AiFSUWlhe0gbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b67722198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAABUCAYAAACSjrEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXlYVVX7978HDlOMDuSAiGY5lhM9PaahOJATkgg4pICB\nKYqGismoojKJE2agmKWPSGGWI6mIkfTI60B2SYppiVNC+dOcAFE4nP3+cV72c/be6+yz6SWU53d/\nrutcdc76nnutvb97rb1YrrNvFcdxHAiCIAiCIAiCkMXkWTeAIAiCIAiCIJoDNHEmCIIgCIIgCAXQ\nxJkgCIIgCIIgFEATZ4IgCIIgCIJQAE2cCYIgCIIgCEIB6mfdAIIgCIIgCIJQyq9vjRS8f+VEbpPV\nTRNngiAIgiAIovlgYvrMqqaJM0EQBEEQBNFsUKmf3fSV9jgTBPFc0L17d3h5eWH8+PHw9vbm/1te\nXm7wOxcuXEBYWFijtSE3Nxf+/v5/6btRUVHYtm2b4LPjx4/Dy8sLo0ePxvz581FVVWW0TO479ZSV\nlSEkJAT9+vXDoUOHAAD79u3Da6+9htTUVDx48AAAkJSUhICAAPz0009/6ZgIgiCeR1TmZoJXU0IT\nZ4IgngtUKhUyMzOxb98+7N27l/9v+/btDX7n1VdfxYYNGxq9HQ2htLQUgYGBOHLkiODze/fuITo6\nGmlpaTh8+DA6dOiANWvWyJbJfUcfJycnjBo1Ci1btsSYMWMAAB4eHjA1NYWfnx8cHBwAAP369cOn\nn36K3r17/5VTQRAE8VyiMjcXvJoSmjgTBPFcwHEcOI5jlp05cwa+vr4IDQ2Fl5cXJk2ahNLSUpw5\ncwbjxo0DADx+/BhhYWHw9vbGhAkTsHTpUv77u3btwrhx4zB+/HgEBwfj+vXrfNmGDRvg4eGBiRMn\nIi8vj/88Pz8fEydOxIQJE/Duu+/i3LlzzLZ9/vnn8PHxwahRowSfFxYWonfv3nB2dgYATJkyBQcP\nHpQtk/uOmBYtWgjeHzx4EG3atMH9+/cBALdu3UKbNm1gZta0qzEEQRB/Nyq1WvBqSmiPM0EQzw0B\nAQEwNdX96IPjODg7O2Pjxo0AgJ9//hnR0dHo378/srOzsXjxYkRERPDfzcvLw+PHj7F3715otVrE\nxcXht99+Q1lZGT777DPs2rULDg4O2Lt3L0JDQ/HNN9/g2LFjOHbsGA4cOAALCwvMmTMHAHDjxg2s\nX78eO3fuhL29Pa5cuYLp06fj2LFjsLS0FLR5yZIlAICTJ08KPv/999/Rtm1b/n3btm1RVVWFqqoq\nZlllZSV+++03g9+xtrYWxK9fVQaAH3/8Ed26dUOrVq34ifOPP/4ILy+vBjpAEATx/NOQ7RkajQbR\n0dEoKytDbW0tQkJCMGzYML48Pz8f6enpUKvV8PHxgZ+fn2w8mjgTBPHckJmZCXt7e2ZZt27d0L9/\nfwCAj48PVqxYgYcPH/Llrq6uSE1Nhb+/PwYNGoTAwEA4OzsjOzsbo0eP5iea3t7eSExMRFlZGU6d\nOgUPDw9YWVnxcTMzM1FYWIg7d+5g+vTp/Cq4Wq3GjRs30K1bN0XHYmj13NTUVLZM6ef1K841NTU4\ne/Ys3n//fTg4OOD+/fs4efIkBgwYwGsrKipw8uRJXLt2DbNmzVLUfoIgiOeVhmzPOHDgAFq0aIGU\nlBQ8fPgQ48eP5yfOGo0GycnJ2LNnDywsLDBlyhQMHz4cLVu2NBiPJs4EQTw3GJpQArqJq1hnYvKf\n3WYdOnTA0aNHcebMGZw6dQqBgYFYsmQJtFqtJJZWq4VGo5HUqb/aPXDgQKxbt44v++OPP9CmTRvF\nx9KuXTsUFxcLvm9nZwdLS0tJ2e3bt2Fvb4/27dsb/I6YFi1agOM4ZGVlwdfXF4BuFbq8vByWlpZ4\n8cUXea2trS169eqFX375RXH7CYIgnlcasj1j9OjR/FY6rVYruJeUlpbCxcUFNjY2AHQLMEVFRRg5\nciQzFkB7nAmCaCZcvHiRn/jt2rUL/fv3h62tLV/+xRdfIDIyEoMGDUJ4eDjc3Nzw66+/ws3NDYcP\nH8a9e/cAAF9//TVatGgBFxcXuLm54ciRI6ioqIBWq8X+/fsBAAMGDEBhYSGuXr0KACgoKMA777yD\np0+fKm7vW2+9hZ9++gk3b97k2zx8+HBmWXZ2NkaMGIFBgwYZ/I4YW1tbVFRUwM7Ojl99btGiBU6c\nOIERI0YobidBEERzoyFP1bCyssILL7yAyspKhIWFYcGCBXxZZWWl4D5ibW2NiooK2Xi04kwQxHOB\nSqWS7HFWqVRYsGABLC0t4ejoiPXr1+PWrVto3bo1UlJScOvWLf7748ePR1FREcaMGQMrKys4OTkh\nMDAQtra2CAwMRGBgIADd5DIjIwMAMGTIEPz666/w8fGBvb09unfvjvv376NLly5YsWIFFi5cCEC3\nEr1p0ybmyq8hWrZsicTERMybNw8ajQbOzs5ISUmRLbOzszP4HRYDBgyAj48P/97R0RHz588XrMQT\nBEH8t9HQJ2n8/vvvmDt3LqZNm8Y/iQgAbGxsUFlZyb+vqqqCnZ2dfN2c3L+NEgRBPAecOXMGK1eu\nNPiECcI4ZWVl2Lt3L+bOnfusm0IQBPH/RdnCaMF7p3WJBrV3795FQEAAli5dKvjtB6Db4zx27Fjs\n3r0blpaWmDx5MjZv3izY6iaGVpwJgiD+y6mqqkJubi4uXLiAX3/9Fa+88sqzbhJBEMRfpiErzhkZ\nGXj06BHS09ORlpYGlUqFiRMnorq6Gn5+foiKikJQUBA4joOfn5/spBmgFWeCIAiCIAiiGfH7kgTB\n+3YrY5qsblpxJgiCIAiCIJoNKrNnN32liTNBEARBEATRbGjqNNv6/O0T59RD38uWzx8zGACwNue4\nrC7c0x1J+741Wl/U+OHYeOSEUd28UW8hIC1LVrMjdCoAwHfdNlndVwvfAwBM/ShTVpf1gT+813xm\ntG17FwVhbPIWWc03kTMBAHFf5crq4nx1zyJc8uVhWd3KiaORdrTQaNtC3x6EVQfyZTURXroHizfm\neRu/+lOjbdv3YTAyjp2U1cwa8SYAYHGW/I/MUqbq0jjP3PKlrG7LzIkI+WS30bZtft9P8TEY0+37\nMBgA8MG2PbK6j96boFintG+9n7FLVvPJrEkAgLDte2V1G6Z7AwAmrJW/RvaEv2fUA0Dnw+TUHbKa\n7PkBAIA5n34lq0sP1j0TWYn3xq43QHfNKT1vxq6lze/rMlpFffGNrC5pylijYwOgGx/CMw8Y1a31\n91Lc75X2QSXex+/Jk9UAQOwED6z/pkBWs2DsEADASiPxlkzwAKDMh9lb5a8jANg0w1fxeZuU+i9Z\n3a75uqfCpBz8Tla3eNxQo/cPQHcPUVqn0vOmZOxSet5mbM42qtsaMhleKVtlNQcWzwAAo9f5Wn9d\nlk0l52Thjv1G27Yu4B1FXgHAst1HZHXL/XTPIF7x9VFZ3VKftxWPl1M2yN93vwjzNxrnWaIyU545\nsLGhFWeCIAiCIAii2dCQBCiNDU2cCYIgCIIgiGaDyuLZbdVQ/JR8VtpagiAIgiAIgmhKVGZmgpcS\niouL4e8v3YKyfft2eHp6IiAgAAEBAbh+/bpsHNkV599++w1JSUm4cOEC1Go1tFotunbtiqioKHTu\n3FlRQwmCIAiCIAiisWjoVo2tW7di//79sLa2lpSVlJQgJSUFPXv2VBRLtuaYmBiEh4ejT58+/Gfn\nzp1DVFQUsrONb9wnCIIgCIIgiMakoU/VcHFxQVpaGhYvXiwpKykpQUZGBu7cuQN3d3fMnDlTNpbs\nVo2amhrBpBkA+vbt26DGEgRBEARBEERjoTI3E7yM4eHhAVNTU2bZ2LFjsXz5cuzYsQNnz55FQYH8\nU3pkV5y7deuGqKgouLm5wdbWFlVVVSgoKEC3bt2MNpIgCIIgCIIgGpvGTIASGBgIGxsbAMCQIUNw\n8eJFDBkyxKBetua4uDgcO3YMZ8+eRWVlJWxsbDB06FB4eHg0WoMJgiAIgiAIQil/9TnOHMcJ3ldW\nVsLT0xOHDx+GpaUlTp06BV9fX/m6OXEUgiAIgiAIgnhOqfhWuJ3CdrjhFeJ6ysrKEB4ejuzsbOTk\n5KC6uhp+fn44cOAAduzYAQsLC7z55puYO3eubJy/feKsNGOWkiw2SjMCGotVH29UYoas5kj0LABQ\nnPlJSUayNUYyJALAIk93xZnXlGbnM5YZLmr8cPit3260bbsXTMfcz+Qz0X0cpMtYpzRrkpJrRGmG\nNqXZICM+z5HVrXrXU7Eudpd8VkYAiJ802uh5A3TnTun5VZqhbdrGnbK6nfOmKc4cqCRzFaA8A96C\nf+2T1a0PHK/4/AZtkv/B8mezJwOA4kxjSjKIGcuMCuiyozZmpkoAiMk+JKtLmDzGaEZOQJeV01hW\nUUCXWdRz1SeympyI9wEoz2aq5BoxliER0GVJVDrWGMtat2mGbqVJSWZGY9neAF3Gtw93ymesWz1N\nl7FOSV8AYDT77N5FQdjy7SmjbZs5fIDizIFKxxolGYCV3p+NnQ9Ad05GJmyW1eTGhACA4nFVyT3V\nmKeAzlel46BSH5SMI0ozLr6X/oWsZtucKUbjPEsqC4SZjm2GDGqyuikBCkEQBEEQBNFsUPKDwL8L\nmjgTBEEQBEEQzYa/use5MaCJM0EQBEEQBNFsoIkzQRAEQRAEQSiAtmoQBEEQBEEQhAIamjmwMZGd\nOPv7+6O2tlbwGcdxUKlUlHKbIAiCIAiCaHL+ylaN4uJirFmzBpmZwie75OfnIz09HWq1Gj4+PvDz\n85ONIztxXrRoEWJjY5GWlmYwVSFBEARBEARBNBUqdcM2TGzduhX79++HtbW14HONRoPk5GTs2bMH\nFhYWmDJlCoYPH46WLVsajGUiV1GfPn3wzjvv4PLly3BychK8CIIgCIIgCKKpeWJrLXgZw8XFBWlp\naZLPS0tL4eLiAhsbG5iZmcHV1RVFRUWysShzIEEQBEEQBNFsqKioELy3tbU1+h39zIH1nD17FllZ\nWVi3bh0A4KOPPkL79u1l027/7T8ONJYpb5GnOwAYzcK03G+U0axEgC4zkdIMg2OTt8hqvomcCQBG\nM+rtXjAdgLIMg8YygwG67GBKMw7F78mT1cVO8ACgLFua0gxt41d/KqvZ92EwAOXZ+ZRkGjOmqdcp\nyZAIKMtwBQDhmfIZotb6eynObmbsfAC6c6IkAyUATFgrn+FqT7gua2RjZkFTmhXOWNa6rA/8ASjL\n5tXY521y6g5ZXfb8AMU6Y30B0PUHpefNWBa/lRNHAwDWf1Mgq1swdojR8wHozomxzJKALruk0gx4\nSscuJeOv0jFfaRbYDYf/LasLG+0GQNn1aywLIaDLRKgkKy4AxedXSZa5xL3HjLYt2nuEYq+UZgBW\nkqXUWKz6eEqzxSody5XMMQBl46rSOYbSMUnp+KDkWlKacVFpX/hvx8bGBpWVlfz7qqoq2NnZyX5H\ndqsGQRAEQRAEQTxP1JiYCV5KEW+y6NKlC27cuIFHjx6hpqYGRUVF6Nu3r2wMehwdQRAEQRAE0Wyo\n0Wj+0vdUKhUAICcnB9XV1fDz80NUVBSCgoLAcRz8/Pzw4osvysZo8MS5pqYG5s/w+XkEQRAEQRDE\n/140Wm2Dv+Pk5MTvb/b09OQ/d3d3h7u7u+I4Brdq5OfnY+jQofDw8MChQ4f4z2fMmNHgxhIEQRAE\nQRBEY1BTWyd4NSUGV5w3b96Mffv2QavVIiwsDE+fPoW3t7dkfwhBEARBEARBNBW1dX9tq0ZjYHDi\nbGZmBnt7ewBAeno6AgMD0a5dO35/CEEQBEEQBEE0NZq6hm/VaCwMbtVwcnJCUlISHj9+DBsbG3z8\n8cdYsWIFrl692pTtIwiCIAiCIAiepxqN4NWUGEyAotFocODAAYwePRpWVlYAgLt37yIjIwMxMTFN\n2kiCIAiCIAiCAIBTV24K3g94uaNBLcdxiIuLw+XLl2Fubo6EhAQ4Ozvz5du3b8dXX33Fp9lesWIF\nOnXqZDAeZQ4kCIIgCIIgmg0nLl8XvH+rWyeD2ry8POTn5yMpKQnFxcXIyMhAeno6X/7hhx/ivffe\nQ8+ePRXV/bc/x3nF10dly5f6vA1AWUacTXn/x2h9sz0GKs4yt9JI1r0l/y/rnrFMPOsDxwNQlp3P\nWLYeQJexJ+Xgd7KaxeOGAlB23gDAI36TrC4vdrbRbFmALmOWsaxU0d4jACjLCgdAUSZCYxkSAV2W\nRCVZ8gBlWcsAGM2mGD9pNBZnHTTatpSp4xQfg9KMi0GbsmV1n82eDABGM3DNGvEmvNd8ZrRtexcF\nKc5UqfT8KslepTR7XGD657Kaf815F4Cy6w1QljkwbPteo23bMN1b8XVp7FpKmToOAOAe97Gs7njc\nXMWZA43Fqo+n5DoCYPRa2rsoCICy8VLpWO67Tj7b21cLdVk0lY5JaUcLZXWhbw9SnAFP6Xip9F75\nXvoXsrptc6YY7X+Arg8qzR6nNAOwkqx7SjMuGrs/A7p7tJKMvYDyTLZK+qrSvvXZ8TOymiD3NwAA\nH+fKZyKcO/ItAMqyGC/csd9o29YFvIPUQ9/LauaPGWw0zrOkIc9xPnv2LNzcdNdynz59cOHCBUF5\nSUkJMjIycOfOHbi7u2PmzJmy8SgBCkEQBEEQBNFsqKlT/gi6yspK2Nra8u/VajW0Wi1MTHQ/8xs7\ndiymTp0KGxsbhIaGoqCgAEOGDDEYj1JuEwRBEARBEM2GWo1W8JLDxsYGVVVV/Hv9STMABAYGwsHB\nAWq1GkOGDMHFixdl4zVo4vzkyRPU1NQ05CsEQRAEQRAE0WjU1mkELzn69++PgoICAMC5c+fQtWtX\nvqyyshKenp6orq4Gx3E4deoUevXqJRtPdqvGlStXsG7dOtjb22PcuHGIjY2FiYkJYmJiMHToUKXH\nRxAEQRAEQRCNQq1G+VYNDw8PFBYWYvJk3e9+kpKSkJOTg+rqavj5+WHhwoXw9/eHhYUF3nzzTQwe\nLL+/W3bivGzZMoSFhaGsrAwffPABcnNzYWFhgRkzZtDEmSAIgiAIgmhyahuwx1mlUmH58uWCzzp3\n7sz/v5eXF7y8vBTHk504a7VavPGG7lefp0+fRqtWrXRfUtNvCgmCIAiCIIimp6YBK86Njewe586d\nOyMmJgZarRbJyckAgC1btqB169ZN0jiCIAiCIAiC0KdWUyd4NSWyCVC0Wi3y8/MxYsQI/rP9+/fj\n7bff5rMJEgRBEARBEERTIX72df2zrpsC2T0XJiYmgkkzALzzzjt/a4MIgiAIgiAIwhBNvcqsD21W\nJgiCIAiCIJoNDUmA0tjQxJkgCIIgCIJoNjRkxZnjOMTFxeHy5cswNzdHQkICnJ2d+fL8/Hykp6dD\nrVbDx8cHfn5+svFo4kwQBEEQBEE0GxryOLpjx46hpqYG2dnZKC4uRlJSEtLT0wEAGo0GycnJ2LNn\nDywsLDBlyhQMHz4cLVu2NBiPUm4TBEEQBEEQzYYaTZ3gJcfZs2fh5uYGAOjTpw8uXLjAl5WWlsLF\nxQU2NjYwMzODq6srioqKZOPRijNBEARBEATRbNA0YMW5srIStra2/Hu1Wg2tVgsTExNJmbW1NSoq\nKmTj0cSZIAiCIAiCaDY0JAGKjY0Nqqqq+Pf1k+b6ssrKSr6sqqoKdnZ2svFoqwZBEARBEATRbGjI\nVo3+/fujoKAAAHDu3Dl07dqVL+vSpQtu3LiBR48eoaamBkVFRejbt69svCZbcTb2q0Z9iouLsWbN\nGmRmZjLLNRoNoqOjUVZWhtraWoSEhGDYsGESnVarRWxsLK5duwYTExMsX74cL7/8MjPmn3/+CR8f\nH2zbtk2Qw1zMhAkTYGNjAwDo0KEDEhMTJZotW7YgPz8ftbW1ePfdd+Hj4yPR7N27F3v27IFKpcLT\np09x6dIlFBYW8rH1jzUiIgJlZWVQq9VYuXIls301NTWIiorCrVu3YGNjg2XLlqFjx458uf45vXnz\nJiIjI2FiYoJXXnkFy5YtY+rqSUpKwksvvYRJkyYxdT///DPi4+NhamoKc3NzpKSk8Bvr9XVXrlzB\n0qVLAQAuLi5ISEiAiYkJs86DBw8iKysL2dnZBuucNWsWOnXqBACYMmUKRo8eLdHdu3cPsbGxqKio\nQF1dHVatWsVfd/q6hQsX4u7du+A4DmVlZejXrx/Wrl0rqTMuLg5qtRqdOnVCQkICs20lJSWIi4uD\nhYUFunfvjtjYWOY1+/LLL0t8kLu29X1g6dq3b4+VK1fyPiQmJmLNmjUCjYuLC5YsWSLwQKvVGqxT\n3wdWne3atZP44OHhIdH17dtX4ENiYiLS09MFmpycHIkHq1atYh7nsmXLBD6w2ta2bVssW7ZM4ANr\nTDA3N5f4IDd21Pvg5+cn0dTW1kr6goODg0QHQNIXABisU98HVttqa2sFPkyaNAmFhYUCTcuWLSV9\nwcnJSRArLi4OmzZtkviwevVqSZ0ajUbiA6ttNTU1Eh8A4ZhrampqcExijc2sMUlf9+TJE+aYpK+p\nq6tjjkeG6mSNSeI6DY1J+jp7e3uDY5K+buPGjcwxSV/z9OlTiQestlVXVzM9EN/PQkJCmD4Yuu+J\nfRDrAgICBGNSSkoKZsyYIdAEBwdLxiQTExODdYp9EOv8/f2ZPoh1ixYtYvog1j158gR37twBAN6H\na9euSY6T5YM41rRp05g+iOcM//jHPyQ+GJpXiD0Q63r16iXxoGXLlhJd3759mT48rzRkq4aHhwcK\nCwsxefJkALpzlpOTg+rqavj5+SEqKgpBQUHgOA5+fn548cUX5QNyTcTRo0e5yMhIjuM47ty5c9zs\n2bOZuk8++YTz9PTkJk2aZDDW119/zSUmJnIcx3EPHjzg3N3dmbq8vDwuOjqa4ziOO336tME6a2tr\nudDQUG7kyJHc1atXDdb79OlTztvb22B5fT0hISEcx3FcVVUVt3HjRlk9x3Hc8uXLuS+//JJZduzY\nMW7+/Pkcx3FcYWEhN2/ePKZu586d3JIlSziO47irV69yQUFBfJn4nIaEhHBFRUUcx3Hc0qVLuby8\nPKbuzz//5GbMmMF5eHhw2dnZBuNNmzaNu3TpEsdxHJednc0lJSUxdXPmzOF++OEHjuM4LjIyksvL\ny2P6XVJSwgUGBgo+E+u+/PJLbtu2bZLzINZFRkZyhw8f5jiO406dOsUdP36cqavn4cOH3Pjx47m7\nd+9KNKGhodz333/PcRzHhYeHc9999x0z1oQJE7hz585xHMdxqamp3IEDBwTX7MOHDzl3d3emD6xr\n+969exIfWPHEPgQFBUk0oaGhEg8M9SexDyzd7t27JT6wdGIfEhMTDfZhfQ9Yxzl37lyuoKBA4ANL\n5+PjI/GBNSawfGDpxP2BpWH1BZaO5YOh8UrsA0sn7g8sDasvyI2R+j6wdHPnzpX0B5aO1R/EY66h\nMUmsMzQm6etKS0uZPohjscYjVp0sD1g6Q2OSWGdoTDJ0H9L3QawxNCaJzwfLA9b9jOUDS8fygaUT\n+xAfHy/RsHwwdK8V+8DSsXxg6Vg+yN3j630oLy+XaFg+sGKxfGDNGcQ+pKenSzQsD1ixWH2BpWON\nSc8zAWlZgldT0mR/Tsj9qlEfFxcXpKWlycYaPXo0wsLCAOhWldVq9sL5iBEjsHLlSgC6vxTt7e2Z\nulWrVmHKlClG/8q4dOkSHj9+jODgYEyfPh3FxcUSzYkTJ9C1a1fMmTMHs2fPxtChQ2Vjnj9/Hleu\nXDH43MBOnTqhrq4OHMehoqICZmZmTN2VK1cwePBgAEDnzp1x9epVvkx8TktKSvD6668DAAYPHoyT\nJ08ydY8fP8a8efPg5eUlqEusW79+Pbp16wZAt0JuYWHB1H388cdwdXVFTU0N7ty5A1tbW4nm/v37\nSE1NRUxMjGydJSUlOH78OKZNm4aYmBg8fvyYqfvxxx/xxx9/4L333kNOTg7++c9/MnX1fPTRR5g2\nbRpatWol0fTo0QP3798Hx3Goqqrirzux7vbt2+jTpw8AoF+/fjh79qzgmq2rq4OpqSkuXrwo8YF1\nbbN8EMdTq9VITU0V+NCtWzeJhuUBq84HDx5IfGDpSkpK8N133wl8EOtMTU0lPoSEhBjsw/oesI6z\nR48eePDggcAH1vn9448/JD7ojwnl5eWwt7dn+sAaO6qrqwU+sDSsvsDSbdy4UeIDS8fygaUT+zBw\n4EDJcbL6gtwYqe8DS8fqD+Lza2dnx+wP+mMux3FMDwDp2Cz2oB59nUqlYvogjsXqC6w6WR6wdIbG\nJLHO0Jhk6D6k74NY07NnT+aYJNaxPGDdz1g+sHQsH1g6sQ8VFRUSDcsHViyWDywdyweWjuWD3D2+\n3oc7d+5INCwfWLFYPojnDO7u7hIf8vLyJPMKlges+QerL7B0rDHpeaamtk7wakqabOJs6FeNYjw8\nPGBqaioby8rKCi+88AIqKysRFhaGBQsWGNSamJggMjISCQkJGDdunKR8z549aNWqFQYNGgSO42Tr\ntbS0RHBwMD799FPExcVh0aJFkmO4f/8+Lly4gI8++ghxcXEIDw+XjbllyxbMnTvXYLm1tTVu3bqF\nUaNGYenSpfD392fqevTogePHjwPQ7eH5n//5H/54xOdU/zj1f0Eq1nXo0AG9e/eW1CXWtW7dGoDu\nhvD5559j+vTpTJ1KpUJ5eTnGjRuHBw8eoHv37gJN/T/zRkZGwsrKStBOcaw+ffpg8eLF2LlzJ5yd\nnbFx40amrqysDA4ODti2bRvatm2LLVu2MHUAcO/ePZw+fRoTJkxgaur/CW7s2LG4d+8e3njjDabO\n2dkZP/zwAwDgu+++Q3V1NfOaZfnA0jk5OUl8YOlatWol8OH9999n9hOxB+JYYWFhiImJkfgg1s2f\nPx+9e/dGRESEwAdW28Q+ZGZmMtsm9oBVZ/0/I+r7wKqT5QPwnzEhPj4enp6eBvuDeOxg+SDWGOoL\nYh2rL4h1Y8eOZfrAitenTx+JD+LjNNQXWGOk2AeWjuWD+PyOGzdO4sOlS5ckY67+OFrvAWtsZnnA\n0ol9aN3xyO1gAAAJFUlEQVS6tUTD8kAcq66ujumBWMdxHHNMYulYPhi6D+n7wIrF8oAVi9UXWPcz\nVl9g6dq1ayfxgaWr37JX74Ovry+zTrEP4lgLFy5EZGSkxAdWnb169ZL4wNLdvHlT4oOhe7y+DyyN\nk5OTxAeWztnZmX/UWb0P4jmDeF5hbW2NyspKybyC1RdY8w/WmMTSGRqTnlc02jrBqylpsj3Ocr9q\n/Cv8/vvvmDt3LqZNm4YxY8bIapOTk/Hnn3/Cz88Phw4dgqWlJV9Wv8+4sLAQly5dQkREBDZt2sRP\nQvTp1KkTXFxc+P93cHDAnTt30KZNG17j4OCALl26QK1Wo3PnzrCwsMC9e/eYD9OuqKjA9evX+RsO\ni+3bt8PNzQ0LFizA7du3ERAQgIMHD8Lc3Fyg8/HxQWlpKaZOnYr+/fujV69eUKlUzJj6513JL0iV\ncOjQIWRkZGDLli1o0aKFQV379u2Rm5uL3bt3IykpCcnJyXxZSUkJbt68ibi4ODx9+hSlpaVISkpC\nVFSUJM6IESP4P8Q8PDwQHx/PrM/BwYFf9R82bBhSU1MNtu3IkSPw9PQ0eN4SEhLw+eefo0uXLsjK\nykJycjK/R1KfxMREJCQkoK6uDq6urvwKvP41O3bsWKxevZr/jr4PSq9tlk7sA0vD8kBf17FjR4M+\niI+hoqKC6YNYl5SUJPGB1TaWB+JYAwcOZPog1vXs2ZPpA/CfMcHX1xdPnz5l+qCvY40dhjT5+fnM\nviDWGeoL9bphw4bB0dHRYH/Qj5ednc2vLur7oH+cdnZ2BvuCuG2G+oL4vBnqD/rxNm3ahNWrV/M+\nlJeXo7CwEIWFhbh8+TIiIiJw//59iQdKx2ZDutOnT/M+RERESDTp6ekSD27duiXQeXl5oUOHDhIP\nSkpKBLrIyEhB2+o9OH/+PK+rP1ZTU1OJD2fOnGEeQ25uLu+D/nHWx/r555+xf/9+gQe//PKLpG2L\nFy/G5s2bBX2BdT+7ePGixAcl9736Mpbu7NmzvA+Ojo547bXXJBqxDytWrBDEKi8vh6mpqcSH8PBw\nSZ2DBw/m21bvA6tt5eXlEh/qfwcibt+3337L+8CKFRkZiZycHIEPkZGREl14eDg2b96M9PR03gcz\nMzPJnOH27dsCH2xsbODm5mZ0XmFo/nHq1CnBmGRIJ3d/ft5oSObAxqbJVpzlftXIQm719+7duwgO\nDsaHH34Ib29vg7r9+/fzqyoWFhYwMTGRTNZ37tyJzMxMZGZmonv37li1ahVz0gwAX3/9NX8h3b59\nG1VVVXB0dBRoXF1d8e9//5vXPHnyxOBEsqioCAMGDDDYfgCwt7fnf1xga2sLjUbDXKk/f/483nzz\nTWRlZWHkyJEGf3gJ6P55r/6v3u+//x6urq6CcmMr72L279+PrKwsZGZmwsnJyaBu9uzZuHHjBgDd\nX9H6XnAch9deew0HDx7Ejh07sG7dOrz88svMSTMABAcH4/z58wCAkydPolevXkydq6srf90VFRVJ\nfhyqf6wnT57kt7uwcHBw4L1o06YNHj16xNQVFBRg7dq12LZtGx48eICBAwcyr9kePXpIfFB6bbN0\nYh9YGpYHYl3v3r2ZPrDisXxg6cQ+tG/fnnmcYg9YsVg+sHQsH1hjwquvvoozZ84IfFAydrA0ubm5\nkr4g1qlUKoSGhkp8EOscHR1x6NAhiQ+sePPmzcNPP/3En0Nzc3NJ215//XX+X6Xq+4Kh4xT7wNI5\nODjA2tpa4ANLd/z4cYEP8fHxgjE3JSUFbm5ukr6gdGxm6U6cOCHwQV/To0cPrFq1CkuWLJF4II51\n6NAhHD16VOIBK97s2bMFHvTq1UsSLyUlBe7u7hIfWLpWrVoJfGBpOnbsKOkLrPNx4cIFSV8Q388q\nKysxaNAgSV9Qct8D2PfH06dPC3xg1bl06VKJD2Jd586dkZubK/GBFW/OnDkSH1i6ESNGSHwwdKz6\nPrBidezYUdIXWLGKi4slPojnDNXV1RgwYIDAh4EDByqaV7DmH99//71kTGLVGRMTY/D+/DzSkKdq\nNDZNtuLM+lWjHIZW/QAgIyMDjx49Qnp6OtLS0qBSqbB161bJKuzbb7+NqKgoTJs2DRqNBjExMRKN\n0joBwNfXF1FRUXj33XdhYmKCxMREycXl7u6OH374Ab6+vuA4DsuWLTMY99q1a7ITXAAIDAxEdHQ0\npk6dCo1Gg/DwcOaql4uLCzZs2IDNmzfDzs5O8OtqMREREViyZAlqa2vRpUsXjBo1SlBu7Dzoo9Vq\nkZiYiPbt2yM0NBQqlQpvvPEGc/vJzJkzERkZCXNzc1hZWQlWiRtSJwDExcVh5cqVMDMzg6OjI1as\nWMHURUREIDY2Fl988QVsbW2xdu1aQbl+vdevX5f1Y+XKlZg/fz7UajXMzc35vZxiXFxcEBgYCCsr\nK/zzn//E4MGDkZCQILlmY2JiEB8fL/AhMTFR0bUt7gNarRZXrlwR+PDkyRNJrAULFkg8UNqfWLqo\nqCgkJiYKfEhNTZXoVq1ahZiYGN6H9u3bSzSffPKJxANWnSwfWLqgoCCJD9XV1YIxITY2Fi+99BJi\nY2MFPjx58sTo2CEeX6KjoxEVFSXpC8HBwZJYLVu2lPhgbW2taLxijWvt2rXDihUreB8SEhIQHx8v\nOM7u3bsjJiYG2dnZfF9Qq9XMOsU+sOp0cHDAggULBD60aNFCck5UKpXEBzGNNSapVCr+qS3GxqRZ\ns2YZHI/qYzVkEWH58uUCD4yNSfo+GKrX2JgUHx8vOybVnzfWmFRbWyu4nyUnJ/NPgNH3QaPRGL3v\nAdL7Y0JCAv8UnHofXF1dUVFRwWvq5wFiH+zt7SV1sq4BcZ1JSUmwsLCQ+GBubi45VkdHR4kPlpaW\nkngmJiYCH1h1arVaSV9wdHSUHMP9+/eZfUF/zhAXF8c/7abeh/DwcKxdu9bovEI8/1i6dCkWLlzI\n7AvieYq1tbVsf3jeaEjK7cZGxTV0eZEgCIIgCIIg/hfyfK/FEwRBEARBEMRzAk2cCYIgCIIgCEIB\nNHEmCIIgCIIgCAXQxJkgCIIgCIIgFEATZ4IgCIIgCIJQAE2cCYIgCIIgCEIBNHEmCIIgCIIgCAX8\nX/lWMHnwR4aGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b6774deb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs4AAABUCAYAAACSjrEgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXlYFFca7t+GBnQAQZS4IKIx45q4kUmMDooLxgVRBGKI\nICrGDRxUlEVQENnEDTcUNNGIGIyJK6NBDWpGrgshj0QxmrhHSBwXVEAUmq77B5earqrT1UWGEJj7\n/Z6nn6S73v7OqXrrLBxP16fiOI4DQRAEQRAEQRCyGP3ZFSAIgiAIgiCIpgBNnAmCIAiCIAhCATRx\nJgiCIAiCIAgF0MSZIAiCIAiCIBRAE2eCIAiCIAiCUID6z64AQRAEQRAEQSjl57+/L3j/17PZDVY2\nTZwJgiAIgiCIpoOR8Z9WNE2cCYIgCIIgiCaDSv3nTV9pjzNBEI2C7t27w83NDRMmTIC7uzv/3+Li\nYr3fuXLlCoKCguqtDtnZ2fD19f1d3w0PD8eOHTvqfOzkyZNwdHTk358+fRpubm4YPXo05s+fj/Ly\ncsl3ioqKMHv2bPTr1w9Hjx4FABw8eBBvvfUWkpOT8fTpUwBAQkICpkyZgh9++OF3nRNBEERjRGVq\nIng1JDRxJgiiUaBSqZCeno6DBw/iwIED/H/bt2+v9ztvvvkm1q9fX+/1qAs3b96En58fvv766zod\nA4A7d+4gKSkJtQlcnzx5giVLlmDz5s04duwYOnTogNWrV0u+Z2dnh1GjRsHGxgZjxowBALi4uMDY\n2BheXl6wtrYGAPTr1w+ffPIJevfuXadzIgiCaMyoTE0Fr4aEtmoQBNEo4DiOn0CKuXjxIpKSktCm\nTRv88ssvaN68OeLj4/H48WOsWLECR44cwYsXLxAeHo579+5BpVLhzTffRExMDABg79692L17N4yN\njdGqVSssXboUnTp1AgCsX78eWVlZaNmyJTp27MiXmZOTg61bt0Kj0aBZs2YICQlB3759JXXbs2cP\nPDw8mBN8uWMVFRUICQlBeHg4goODAQC5ubno3bs37O3tAQDe3t4YP348oqKiJN9v2bKl4P2RI0fQ\npk0blJSUwM7ODvfv30ebNm1gYtKwqzEEQRB/NH/mVg2aOBME0WiYMmUKjI1rfvTBcRzs7e2xceNG\nAMCPP/6IJUuWoH///sjMzERISAhCQ0P57544cQIvXrzAgQMHoNVqER0djV9++QVFRUX49NNPsXfv\nXlhbW+PAgQMICAjAP//5T5w8eRInT57E4cOHYWZmhrlz5wIA7t69i3Xr1mH37t2wsrLCjRs3MHXq\nVJw8eRLNmjUT1Hnp0qUAgHPnzknOR+5YVFQUvL290bVrV/6zX3/9FW3btuXft23bFuXl5SgvL4e5\nubng+7WrygDw/fffo1u3bmjVqhVKSkr4z9zc3AxdcoIgiCZHQ2/P0IUmzgRBNBrS09NhZWXFPNat\nWzf0798fAODh4YGYmBg8e/aMP+7o6Ijk5GT4+vpi0KBB8PPzg729PTIzMzF69Gh+ounu7o74+HgU\nFRXh/PnzcHFxQfPmzfm46enpyM3NxcOHDzF16lR+FVytVuPu3bvo1q3bf32eGRkZUKvVcHd3x/37\n9/nP9a241/4xoUvtinNlZSXy8/Px8ccfw9raGiUlJTh37hwGDBjAa0tLS3Hu3Dncvn0bs2bN+q/r\nTxAE8WdSl+0ZGo0GS5YsQVFREaqqqjB79mwMGzaMP56Tk4OUlBSo1Wp4eHjAy8tLNh5NnAmCaDTo\nmzgCNRNXsc7I6D8/0+jQoQOOHz+Oixcv4vz58/Dz88PSpUuh1WolsbRaLTQajaRM3dXugQMHYu3a\ntfyx3377DW3atPmdZybk4MGDePnyJdzd3VFZWcn//9SpU1FQUCAos0WLFpJVbqBm4sxxHDIyMuDp\n6QmgZhW6uLgYzZo1w2uvvcZrLS0t0atXL/z000/1Un+CIIg/k7ps1Th8+DBatmyJpKQkPHv2DBMm\nTOAnzhqNBomJidi/fz/MzMzg7e2N4cOHw8bGRm88+nEgQRBNgqtXr/ITv71796J///6wtLTkj3/+\n+ecICwvDoEGDEBwcDCcnJ/z8889wcnLCsWPH8OTJEwDAV199hZYtW8LBwQFOTk74+uuvUVpaCq1W\ni0OHDgEABgwYgNzcXNy6dQsAcObMGYwfPx6vXr2ql3PZt28fjhw5ggMHDiAtLQ1mZmY4cOAAnJyc\n8MMPP+DevXv8eQ4fPpwZw9LSEqWlpWjRogW/+tyyZUucPXsWI0aMqJd6EgRBNEbq8lSN0aNH809f\n0mq1gkWYmzdvwsHBARYWFjAxMYGjoyPy8vJk49GKM0EQjQKVSiXZ46xSqbBgwQI0a9YMtra2WLdu\nHe7fv4/WrVsjKSlJsM1hwoQJyMvLw5gxY9C8eXPY2dnBz88PlpaW8PPzg5+fH4CayWVqaioAYMiQ\nIfj555/h4eEBKysrdO/eHSUlJejSpQtiYmKwcOFCADUr0Vu2bGGu/NbXuQOAjY0N4uPjMW/ePGg0\nGtjb2yMpKUnv9wYMGAAPDw/+va2tLebPny9YiScIgvhfoy5bNWq34pWVlSEoKAgLFizgj5WVlQkW\nYMzNzVFaWipfNif3b6MEQRCNgIsXL/JPzyB+H0VFRThw4AACAwP/7KoQBEH8VxQtXCJ4b7c2Xlb/\n66+/IjAwED4+PnB3d+c/v379OtasWYO0tDQANc++d3R0xMiRI/XGomUJgiCI/3HKy8uRnZ2NK1eu\n4Oeff/6zq0MQBPFfUZfnOD969Aj+/v5YvHixYNIMAF26dMHdu3fx/PlzVFZWIi8vj/nYUUHZtOJM\nEARBEARBNBV+XRoneN9uRYRebVxcHI4dO4bXX3+d3wL4wQcfoKKiAl5eXjh9+jQ2bdoEjuPg6ekJ\nb29v2bJp4kwQBEEQBEE0GX6LWSl433ZZqB5l/UM/DiQIgiAIgiCaDA2dZluXP3zivP7Yv2SPB412\nAgCs++cZWd2CsUOQdOSUwfJCxg3F5uO5BnUBIwdh+pZMWc2ncz4EAExK/kxWt3d+za/1p2zOkNXt\nCpgMr3U7DdZt34KpcEvaLqs5HDIDABDz1XFZ3TKPmg3uUfu+ltUt9xqF1JPS7GZiZo14z6APIeOG\nAqjf6+a5dofBun25cBq251yQ1cwY9i4AIPzzf8rqErzHAgDmfvKlrC7F39Ogplan9BwmrpHX7Q+e\nBgBY8NlBWd06vwmKdSsP5xisW6jbMMzetk9Ws/VjrzrVzVB72LdgqsEya8udvCFdVpPxD18AQNDO\nA7K69VNr9sDN2S7v65YZnvjk1EWDdfMf+o6iWAAQ+Ol+Wd2m6RMBAJF7j8nqYieNNtg3ADX9Q0iG\n4R9cJk0eh9VZp2U1i1ydAQBp35yX1c0cXpOYRUn/kHDwG4N1C58wXPE4E7v/hKwucqILAGXeG/IK\nqPFL6XXzXi9//34eVHP/rjEQL9jV2eD4AdSMIUrbTPyBk7K6Je41jz801Md9uXAa/rHD8HXbMG0i\nPk7da1C3bdYkRWUCMHifJ00eB0CZD4t3HzZYt1U+bormNYDycVzJ/at0PFIy7jZmVCaUOZAgCIIg\nCIIgDFKXBCj1DU2cCYIgCIIgiCaDyuzP26qh+HF0rLS1BEEQBEEQBNGQqExMBC8lFBQUwNfXV/L5\nzp074erqiilTpmDKlCm4c+eObBzZFedffvkFCQkJuHLlCtRqNbRaLbp27Yrw8HB07txZUUUJgiAI\ngiAIor6o61aN7du349ChQzA3N5ccKywsRFJSEnr27KkolmzJERERCA4ORp8+ffjPLl26hPDwcGRm\nyv+wjiAIgiAIgiDqm7o+VcPBwQGbN29GSEiI5FhhYSFSU1Px8OFDODs7Y+bMmbKxZLdqVFZWCibN\nAAxmVCEIgiAIgiCIPwqVqYngZQgXFxcYGxszj40dOxbLly/Hrl27kJ+fjzNn5J+GIrvi3K1bN4SH\nh8PJyQmWlpYoLy/HmTNn0K1bN4OVJAiCIAiCIIj6RmVSf8+28PPzg4WFBQBgyJAhuHr1KoYMGaJX\nL1tydHQ0Tp48ifz8fJSVlcHCwgJDhw6Fi4tLvVWYIAiCIAiCIJTye5/jLE6WXVZWBldXVxw7dgzN\nmjXD+fPn4enpKV82pdwmCIIgCIIgmgql3wi3U1gO179CXEtRURGCg4ORmZmJrKwsVFRUwMvLC4cP\nH8auXbtgZmaG9957D4GBgbJx/vCJ84yt8j8i3D67Jjufkiw2SjMC+mzcbVC3e54P3Fd/Kqs5sGg6\nACjO/KQk65ehWLXxFu46JKtZO2U8AODD5F2yusz5UwDAYGa4ULdhBjMmATVZkwxlftowrSa7mdKs\nSfWZoU1pNsiIzKOyurgPxwBQlnHRkKZWF5xuONvUGl83xdfXUKbHWSPeAwCDGbi2zZqkOHOgkvsI\nAGamfSGrS5v5AQBlWfwMZXkEajI9Ks1qqDTTmKF+ZPc8H8XZNpVk+gOUZ1xc+oV8vBUfjIZfyh6D\ndfts7kcG61ZbP6V9jZI2AyjLyqm0bkqy6QFQ3LZC92TJ6lZ+5IroL7MN1i3a833FfY1S75X4oHSs\nNHSPfDb3IwDKs0Eq8WFayucG67ZjrrfBMRCoGQeVjuNKM4Yq6R+UZttUOo4r9UFJxlulmVaVZjNt\nrJSdEd7jFkMGNVjZlACFIAiCIAiCaDIo+UHgHwVNnAmCIAiCIIgmw+/d41wf0MSZIAiCIAiCaDLQ\nxJkgCIIgCIIgFEBbNQiCIAiCIAhCAXXNHFifyE6cfX19UVVVJfiM4zioVCpKuU0QBEEQBEE0OL9n\nq0ZBQQFWr16N9HThE8RycnKQkpICtVoNDw8PeHl5ycaRnTgvWrQIkZGR2Lx5s95UhQRBEARBEATR\nUKjUddswsX37dhw6dAjm5uaCzzUaDRITE7F//36YmZnB29sbw4cPh42Njd5YRnIF9enTB+PHj8f1\n69dhZ2cneBEEQRAEQRBEQ/PS0lzwMoSDgwM2b94s+fzmzZtwcHCAhYUFTExM4OjoiLy8PNlYlDmQ\nIAiCIAiCaDKUlpYK3ltaWhr8jm7mwFry8/ORkZGBtWvXAgA2bNiA9u3by6bd/sN/HJh89FvZ4/PH\nDAYAg1mYoj3fN5iJDqjJRrcp+6xBXeD7f4db0nZZzeGQGQCUZ+dLOnJKVhcybqjBbD1ATcYeJVm1\nACDh4DeyuvAJwwEAE9fIZ0PaHzxNcQY8pZnXlGbMivnquKxumcdIg5pandLMdkqzGhrK9rfG101x\ndjOlGfCUZjfzWrdTVrdvwVQAyrJNKb2+SrwClGUCBZRlS1N6fZVet8kb5DNkZvzDV7HOUNYyoCZz\nmdLrpiQjIKDs/jV0PYCaa6I0k5uhbGlJk8cBUJYREICi/tfQ+AHUjCFjEtJkNUfDZwIANn4tPzbM\nG/V3ADB4TXbM9TaYKRaoyRarJAMloCxbIQCDGV4/D/I1OC4ANWOD0uxxSrNyKrluhrKKAjWZRbfn\nXDComzHsXcVZI5W2wUnJn8nq9s73UzzHUJJNuC51U9KvKs1Qq7Qt/K9jYWGBsrIy/n15eTlatGgh\n+x3ZrRoEQRAEQRAE0ZioNDIRvJQi3mTRpUsX3L17F8+fP0dlZSXy8vLQt29f2Rj0ODqCIAiCIAii\nyVCp0fyu76lUKgBAVlYWKioq4OXlhfDwcEyfPh0cx8HLywuvvfaabIw6T5wrKyth+ic+P48gCIIg\nCIL4/xeNVlvn79jZ2fH7m11dXfnPnZ2d4ezsrDiO3q0aOTk5GDp0KFxcXHD06H/2qs6YMaPOlSUI\ngiAIgiCI+qCyqlrwakj0rjhv3boVBw8ehFarRVBQEF69egV3d3fJ/hCCIAiCIAiCaCiqqn/fVo36\nQO/E2cTEBFZWVgCAlJQU+Pn5oV27dvz+EIIgCIIgCIJoaDTVdd+qUV/o3aphZ2eHhIQEvHjxAhYW\nFti0aRNiYmJw69athqwfQRAEQRAEQfC80mgEr4ZEbwIUjUaDw4cPY/To0WjevDkA4NGjR0hNTUVE\nRESDVpIgCIIgCIIgAOD8jXuC9wPe6KhXy3EcoqOjcf36dZiamiIuLg729vb88Z07d+LLL7/k02zH\nxMSgU6dOeuNR5kCCIAiCIAiiyXD2+h3B+79366RXe+LECeTk5CAhIQEFBQVITU1FSkoKf3zx4sWY\nNm0aevbsqajsP/w5zqsNZPVZ9P+y+ijJ0rX5eK7B8gJGDkLs/hMGdZETXQxmJIudVJOla+GuQ7K6\ntVPGA1CWnU/pOdTndQMA15XbZHVZoR8bzHAF1GS5UpqpSWkGsQmrPpHVHVzsbzCzJFCTXVJptkIl\nZQLA4t3yWZhW+bgpzgioNDOj0uxxhjJwpc38AICyLHOGsmUBNRmzDJ3Dcq9RAJRnNTSUHWzGsHex\n5cT/MVi3OS4D4ZeyR1bz2dyPACjLWAcoy9D2cepeg3XbNmuS4qx7SrMfOkdvktWdjg5UnNnOUNY9\noCbzntI+Scl1A5S1QaUZLZVmM1Waye2zb7+T1fkNfttg/wbU9HErDIxHSye6AFCeBVZJJkJD7Q+o\naYP1nWm1PtuMobrV1k9pn6R0bFDil9KsnIbG+4CRgwBA8ZiqZLw3NF8BauYsSu+3xkpdnuOcn58P\nJycnAECfPn1w5coVwfHCwkKkpqbi4cOHcHZ2xsyZM2XjUQIUgiAIgiAIoslQWa38EXRlZWWwtLTk\n36vVami1WhgZ1fzMb+zYsZg8eTIsLCwQEBCAM2fOYMiQIXrjUcptgiAIgiAIoslQpdEKXnJYWFig\nvLycf687aQYAPz8/WFtbQ61WY8iQIbh69apsvDpNnF++fInKysq6fIUgCIIgCIIg6o2qao3gJUf/\n/v1x5kzNdqNLly6ha9eu/LGysjK4urqioqICHMfh/Pnz6NWrl2w82a0aN27cwNq1a2FlZYVx48Yh\nMjISRkZGiIiIwNChQ5WeH0EQBEEQBEHUC1Ua5Vs1XFxckJubiw8//BAAkJCQgKysLFRUVMDLywsL\nFy6Er68vzMzM8N5772Hw4MGy8WQnzlFRUQgKCkJRURH+8Y9/IDs7G2ZmZpgxYwZNnAmCIAiCIIgG\np6oOe5xVKhWWL18u+Kxz5878/7u5ucHNzU1xPNmJs1arxTvvvAMAuHDhAlq1alXzJTX9ppAgCIIg\nCIJoeCrrsOJc38juce7cuTMiIiKg1WqRmJgIAEhLS0Pr1q0bpHIEQRAEQRAEoUuVplrwakhkE6Bo\ntVrk5ORgxIgR/GeHDh3CyJEj+WyCBEEQBEEQBNFQbMo+K3gf+P7fG6xs2T0XRkZGgkkzAIwfP/4P\nrRBBEARBEARB6KOhV5l1oc3KBEEQBEEQRJOhLglQOI5DdHQ0rl+/DlNTU8TFxcHe3p4/npOTg5SU\nFKjVanh4eMDLy0s2Hk2cCYIgCIIgiCZDXVacT548icrKSmRmZqKgoAAJCQlISUkBAGg0GiQmJmL/\n/v0wMzODt7c3hg8fDhsbG73xKHMgQRAEQRAE0WSoqq4WvOTIz8+Hk5MTAKBPnz64cuUKf+zmzZtw\ncHCAhYUFTExM4OjoiLy8PNl4tOJMEARBEARBNBnq8ji6srIyWFpa8u/VajWfdlt8zNzcHKWlpbLx\naOJMEARBEARBNBk0ddjjbGFhgfLycv597aS59lhZWRl/rLy8HC1atJCNR1s1CIIgCIIgiCZDpaZa\n8JKjf//+OHPmDADg0qVL6Nq1K3+sS5cuuHv3Lp4/f47Kykrk5eWhb9++svFoxZkgCIIgCIJoMtRl\nq4aLiwtyc3Px4YcfAgASEhKQlZWFiooKeHl5ITw8HNOnTwfHcfDy8sJrr70mG082AUp9YuhxILoU\nFBRg9erVSE9PZx7XaDRYsmQJioqKUFVVhdmzZ2PYsGESnVarRWRkJG7fvg0jIyMsX74cb7zxBjPm\n48eP4eHhgR07dghymIuZOHEiLCwsAAAdOnRAfHy8RJOWloacnBxUVVXho48+goeHh0Rz4MAB7N+/\nHyqVCq9evcK1a9eQm5vLx9Y919DQUBQVFUGtVmPFihXM+lVWViI8PBz379+HhYUFoqKi0LFjR/64\n7jW9d+8ewsLCYGRkhL/+9a+Iiopi6mpJSEjA66+/jkmTJjF1P/74I2JjY2FsbAxTU1MkJSXxv0jV\n1d24cQPLli0DADg4OCAuLg5GRkbMMo8cOYKMjAxkZmbqLXPWrFno1KkTAMDb2xujR4+W6J48eYLI\nyEiUlpaiuroaK1eu5O87Xd3ChQvx6NEjcByHoqIi9OvXD2vWrJGUGR0dDbVajU6dOiEuLo5Zt8LC\nQkRHR8PMzAzdu3dHZGQk85594403JD7I3du6PrB07du3x4oVK3gf4uPjsXr1aoHGwcEBS5cuFXig\n1Wr1lqnrA6vMdu3aSXxwcXGR6Pr27SvwIT4+HikpKQJNVlaWxIOVK1cyzzMqKkrgA6tubdu2RVRU\nlMAHVp9gamoq8UGu76j1wcvLS6KpqqqStAVra2uJDoCkLQDQW6auD6y6VVVVCXyYNGkScnNzBRob\nGxtJW7CzsxPEio6OxpYtWyQ+rFq1SlKmRqOR+MCqW2VlpcQHQNjnGhsb6+2TWH0zq0/S1b18+ZLZ\nJ+lqqqurmf2RvjJZfZK4TH19kq7OyspKb5+kq9u4cSOzT9LVvHr1SuIBq24VFRVMD8Tj2ezZs5k+\n6Bv3xD6IdVOmTBH0SUlJSZgxY4ZA4+/vL+mTjIyM9JYp9kGs8/X1Zfog1i1atIjpg1j38uVLPHz4\nEAB4H27fvi05T5YP4lg+Pj5MH8Rzhr/97W8SH/TNK8QeiHW9evWSeGBjYyPR9e3bl+lDY+Xj1L2C\n99tmTdKj/APgGojjx49zYWFhHMdx3KVLl7g5c+Ywddu2beNcXV25SZMm6Y311VdfcfHx8RzHcdzT\np085Z2dnpu7EiRPckiVLOI7juAsXLugts6qqigsICODef/997tatW3rLffXqFefu7q73eG05s2fP\n5jiO48rLy7mNGzfK6jmO45YvX8598cUXzGMnT57k5s+fz3Ecx+Xm5nLz5s1j6nbv3s0tXbqU4ziO\nu3XrFjd9+nT+mPiazp49m8vLy+M4juOWLVvGnThxgql7/PgxN2PGDM7FxYXLzMzUG8/Hx4e7du0a\nx3Ecl5mZySUkJDB1c+fO5b777juO4zguLCyMO3HiBNPvwsJCzs/PT/CZWPfFF19wO3bskFwHsS4s\nLIw7duwYx3Ecd/78ee706dNMXS3Pnj3jJkyYwD169EiiCQgI4L799luO4zguODiYO3XqFDPWxIkT\nuUuXLnEcx3HJycnc4cOHBffss2fPOGdnZ6YPrHv7yZMnEh9Y8cQ+TJ8+XaIJCAiQeKCvPYl9YOn2\n7dsn8YGlE/sQHx+vtw3resA6z8DAQO7MmTMCH1g6Dw8PiQ+sPoHlA0snbg8sDastsHQsH/T1V2If\nWDpxe2BpWG1Bro/U9YGlCwwMlLQHlo7VHsR9rr4+SazT1yfp6m7evMn0QRyL1R+xymR5wNLp65PE\nOn19kr5xSNcHsUZfnyS+HiwPWOMZyweWjuUDSyf2ITY2VqJh+aBvrBX7wNKxfGDpWD7IjfG1PhQX\nF0s0LB9YsVg+sOYMYh9SUlIkGpYHrFistsDSsfqkxsyUzRmCV0PSYH9OyD0ORBcHBwds3rxZNtbo\n0aMRFBQEoGZVWa1m7zgZMWIEVqxYAaDmL0UrKyumbuXKlfD29ja4PH/t2jW8ePEC/v7+mDp1KgoK\nCiSas2fPomvXrpg7dy7mzJmDoUOHysa8fPkybty4ofeB2506dUJ1dTU4jkNpaSlMTEyYuhs3bmDw\n4MEAgM6dO+PWrVv8MfE1LSwsxNtvvw0AGDx4MM6dO8fUvXjxAvPmzYObm5ugLLFu3bp16NatG4Ca\nFXIzMzOmbtOmTXB0dERlZSUePnwIS0tLiaakpATJycmIiIiQLbOwsBCnT5+Gj48PIiIi8OLFC6bu\n+++/x2+//YZp06YhKysL7777LlNXy4YNG+Dj44NWrVpJND169EBJSQk4jkN5eTl/34l1Dx48QJ8+\nfQAA/fr1Q35+vuCera6uhrGxMa5evSrxgXVvs3wQx1Or1UhOThb40K1bN4mG5QGrzKdPn0p8YOkK\nCwtx6tQpgQ9inbGxscSH2bNn623Duh6wzrNHjx54+vSpwAfW9f3tt98kPuj2CcXFxbCysmL6wOo7\nKioqBD6wNKy2wNJt3LhR4gNLx/KBpRP7MHDgQMl5stqCXB+p6wNLx2oP4uvbokULZnvQ7XM5jmN6\nAEj7ZrEHtejqVCoV0wdxLFZbYJXJ8oCl09cniXX6+iR945CuD2JNz549mX2SWMfygDWesXxg6Vg+\nsHRiH0pLSyUalg+sWCwfWDqWDywdywe5Mb7Wh4cPH0o0LB9YsVg+iOcMzs7OEh9OnDghmVewPGDN\nP1htgaVj9UmNmcqqasGrIWmwibO+x4GIcXFxgbGxsWys5s2b4y9/+QvKysoQFBSEBQsW6NUaGRkh\nLCwMcXFxGDdunOT4/v370apVKwwaNAicgV0rzZo1g7+/Pz755BNER0dj0aJFknMoKSnBlStXsGHD\nBkRHRyM4OFg2ZlpaGgIDA/UeNzc3x/379zFq1CgsW7YMvr6+TF2PHj1w+vRpADWb3//973/z5yO+\nprrnqfvoFbGuQ4cO6N27t6Qssa5169YAagaEPXv2YOrUqUydSqVCcXExxo0bh6dPn6J79+4CTe0/\n84aFhaF58+aCeopj9enTByEhIdi9ezfs7e2xceNGpq6oqAjW1tbYsWMH2rZti7S0NKYOAJ48eYIL\nFy5g4sSJTE3tP8GNHTsWT548wTvvvMPU2dvb47vvvgMAnDp1ChUVFcx7luUDS2dnZyfxgaVr1aqV\nwIePP/6Y2U7EHohjBQUFISIiQuKDWDd//nz07t0boaGhAh9YdRP7kJ6ezqyb2ANWmbX/jKjrA6tM\nlg/Af/qE2NhYuLq66m0P4r6D5YNYo68tiHWstiDWjR07lukDK16fPn0kPojPU19bYPWRYh9YOpYP\n4us7btxgq+wbAAAJKUlEQVQ4iQ/Xrl2T9Lm6/WitB6y+meUBSyf2oXXr1hINywNxrOrqaqYHYh3H\nccw+iaVj+aBvHNL1gRWL5QErFqstsMYzVltg6dq1ayfxgaWr3bJX64OnpyezTLEP4lgLFy5EWFiY\nxAdWmb169ZL4wNLdu3dP4oO+MV7XB5bGzs5O4gNLZ29vzz8juNYH8ZxBPK8wNzdHWVmZZF7Bagus\n+QerT2Lp9PVJjRWNtlrwakga7MeBco8D+T38+uuvCAwMhI+PD8aMGSOrTUxMxOPHj+Hl5YWjR4+i\nWbNm/LHafca5ubm4du0aQkNDsWXLFn4SokunTp3g4ODA/7+1tTUePnyINm3a8Bpra2t06dIFarUa\nnTt3hpmZGZ48ecLMQlNaWoo7d+7wAw6LnTt3wsnJCQsWLMCDBw8wZcoUHDlyBKampgKdh4cHbt68\nicmTJ6N///7o1asXVCoVM6budVfy6BUlHD16FKmpqUhLS0PLli316tq3b4/s7Gzs27cPCQkJSExM\n5I8VFhbi3r17iI6OxqtXr3Dz5k0kJCQgPDxcEmfEiBH8H2IuLi6IjY1llmdtbc2v+g8bNgzJycl6\n6/b111/D1dVV73WLi4vDnj170KVLF2RkZCAxMZHfI6lLfHw84uLiUF1dDUdHR34FXveeHTt2LFat\nWsV/R9cHpfc2Syf2gaVheaCr69ixo14fxOdQWlrK9EGsS0hIkPjAqhvLA3GsgQMHMn0Q63r27Mn0\nAfhPn+Dp6YlXr14xfdDVsfoOfZqcnBxmWxDr9LWFWt2wYcNga2urtz3oxsvMzORXF3V90D3PFi1a\n6G0L4rrpawvi66avPejG27JlC1atWsX7UFxcjNzcXOTm5uL69esIDQ1FSUmJxAOlfbM+3YULF3gf\nQkNDJZqUlBSJB/fv3xfo3Nzc0KFDB4kHhYWFAl1YWJigbrUeXL58mdfVnquxsbHEh4sXLzLPITs7\nm/dB9zxrY/344484dOiQwIOffvpJUreQkBBs3bpV0BZY49nVq1clPigZ92qPsXT5+fm8D7a2tnjr\nrbckGrEPMTExgljFxcUwNjaW+BAcHCwpc/DgwXzdan1g1a24uFjiQ+3vQMT1++abb3gfWLHCwsKQ\nlZUl8CEsLEyiCw4OxtatW5GSksL7YGJiIpkzPHjwQOCDhYUFnJycDM4r9M0/zp8/L+iT9OnkxufG\nRl0yB9Y3DbbiLPc4EBZyq7+PHj2Cv78/Fi9eDHd3d726Q4cO8asqZmZmMDIykkzWd+/ejfT0dKSn\np6N79+5YuXIlc9IMAF999RV/Iz148ADl5eWwtbUVaBwdHfGvf/2L17x8+VLvRDIvLw8DBgzQW38A\nsLKy4n9cYGlpCY1Gw1ypv3z5Mt577z1kZGTg/fff1/vDS6Dmn/dq/+r99ttv4ejoKDhuaOVdzKFD\nh5CRkYH09HTY2dnp1c2ZMwd3794FUPNXtK4XHMfhrbfewpEjR7Br1y6sXbsWb7zxBnPSDAD+/v64\nfPkyAODcuXPo1asXU+fo6Mjfd3l5eZIfh+qe67lz5/jtLiysra15L9q0aYPnz58zdWfOnMGaNWuw\nY8cOPH36FAMHDmTesz169JD4oPTeZunEPrA0LA/Eut69ezN9YMVj+cDSiX1o37498zzFHrBisXxg\n6Vg+sPqEN998ExcvXhT4oKTvYGmys7MlbUGsU6lUCAgIkPgg1tna2uLo0aMSH1jx5s2bhx9++IG/\nhqamppK6vf322/y/StW2BX3nKfaBpbO2toa5ubnAB5bu9OnTAh9iY2MFfW5SUhKcnJwkbUFp38zS\nnT17VuCDrqZHjx5YuXIlli5dKvFAHOvo0aM4fvy4xANWvDlz5gg86NWrlyReUlISnJ2dJT6wdK1a\ntRL4wNJ07NhR0hZY1+PKlSuStiAez8rKyjBo0CBJW1Ay7gHs8fHChQsCH1hlLlu2TOKDWNe5c2dk\nZ2dLfGDFmzt3rsQHlm7EiBESH/Sdq64PrFgdO3aUtAVWrIKCAokP4jlDRUUFBgwYIPBh4MCBiuYV\nrPnHt99+K+mTWGVGREToHZ8bI3V5HF1902ArzqzHgcihb9UPAFJTU/H8+XOkpKRg8+bNUKlU2L59\nu2QVduTIkQgPD4ePjw80Gg0iIiIkGqVlAoCnpyfCw8Px0UcfwcjICPHx8ZKby9nZGd999x08PT3B\ncRyioqL0xr19+7bsBBcA/Pz8sGTJEkyePBkajQbBwcHMVS8HBwesX78eW7duRYsWLQS/rhYTGhqK\npUuXoqqqCl26dMGoUaMExw1dB120Wi3i4+PRvn17BAQEQKVS4Z133mFuP5k5cybCwsJgamqK5s2b\nC1aJ61ImAERHR2PFihUwMTGBra0tYmJimLrQ0FBERkbi888/h6WlJdasWSM4rlvunTt3ZP1YsWIF\n5s+fD7VaDVNTU34vpxgHBwf4+fmhefPmePfddzF48GDExcVJ7tmIiAjExsYKfIiPj1d0b4vbgFar\nxY0bNwQ+vHz5UhJrwYIFEg+UtieWLjw8HPHx8QIfkpOTJbqVK1ciIiKC96F9+/YSzbZt2yQesMpk\n+cDSTZ8+XeJDRUWFoE+IjIzE66+/jsjISIEPL1++NNh3iPuXJUuWIDw8XNIW/P39JbFsbGwkPpib\nmyvqr1j9Wrt27RATE8P7EBcXh9jYWMF5du/eHREREcjMzOTbglqtZpYp9oFVprW1NRYsWCDwoWXL\nlpJrolKpJD6Iqa8+SaVS8U9tMdQnzZo1S29/VBurLosIy5cvF3hgqE/S9UFfuYb6pNjYWNk+qfa6\nsfqkqqoqwXiWmJjIPwFG1weNRmNw3AOk42NcXBz/FJxaHxwdHVFaWspraucBYh+srKwkZbLuAXGZ\nCQkJMDMzk/hgamoqOVdbW1uJD82aNZPEMzIyEvjAKlOr1Uragq2treQcSkpKmG1Bd84QHR3NP+2m\n1ofg4GCsWbPG4LxCPP9YtmwZFi5cyGwL4nmKubm5bHtobBhKs/1H0mCPoyMIgiAIgiCIpkzjXosn\nCIIgCIIgiEYCTZwJgiAIgiAIQgE0cSYIgiAIgiAIBdDEmSAIgiAIgiAUQBNngiAIgiAIglAATZwJ\ngiAIgiAIQgE0cSYIgiAIgiAIBfxfg7EV+onn8oUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b7f321b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "TOTAL_EPISODES = 2000 if isFast else 3000\n",
    "\n",
    "def run(agent):\n",
    "    s = env.reset()\n",
    "    R = 0\n",
    "\n",
    "    while True:\n",
    "        # Uncomment the line below to visualize the cartpole\n",
    "        # env.render()\n",
    "\n",
    "        # CNTK: explicitly setting to float32\n",
    "        a = agent.act(s.astype(np.float32))\n",
    "\n",
    "        s_, r, done, info = env.step(a)\n",
    "\n",
    "        if done: # terminal state\n",
    "            s_ = None\n",
    "\n",
    "        agent.observe((s, a, r, s_))\n",
    "        agent.replay()\n",
    "\n",
    "        s = s_\n",
    "        R += r\n",
    "\n",
    "        if done:\n",
    "            return R\n",
    "\n",
    "agent = Agent()\n",
    "\n",
    "episode_number = 0\n",
    "reward_sum = 0\n",
    "while episode_number < TOTAL_EPISODES:\n",
    "    reward_sum += run(agent)\n",
    "    episode_number += 1\n",
    "    if episode_number % BATCH_SIZE_BASELINE == 0:\n",
    "        print('Episode: %d, Average reward for episode %f.' % (episode_number,\n",
    "                                                               reward_sum / BATCH_SIZE_BASELINE))\n",
    "        if episode_number%200==0:\n",
    "            plot_weights([(agent.brain.params['W1'], 'Episode %i $W_1$'%episode_number)], figsize=(14,5))\n",
    "        if reward_sum / BATCH_SIZE_BASELINE > REWARD_TARGET:\n",
    "            print('Task solved in %d episodes' % episode_number)\n",
    "            plot_weights([(agent.brain.params['W1'], 'Episode %i $W_1$'%episode_number)], figsize=(14,5))\n",
    "            break\n",
    "        reward_sum = 0\n",
    "agent.brain.model.save('dqn.mod')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you run it, you should see something like\n",
    "```\n",
    "Episode: 20, Average reward for episode 20.700000.\n",
    "Episode: 40, Average reward for episode 20.150000.\n",
    "Episode: 60, Average reward for episode 21.100000.\n",
    "...\n",
    "Episode: 960, Average reward for episode 20.150000.\n",
    "Episode: 980, Average reward for episode 26.700000.\n",
    "Episode: 1000, Average reward for episode 32.900000.\n",
    "Task solved in 1000 episodes\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 1.1**\n",
    "Rewrite the model without using the layer lib.\n",
    "\n",
    "**Task 1.2**\n",
    "Play with different [learners](https://cntk.ai/pythondocs/cntk.learner.html#module-cntk.learner). Which one works better? Worse? Think about which parameters you would need to adapt when switching from one learner to the other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the DQN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[2017-05-16 20:21:24,188] Making new env: CartPole-v0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "env = gym.make('CartPole-v0')\n",
    "\n",
    "num_episodes = 10  # number of episodes to run\n",
    "\n",
    "modelPath = 'dqn.mod'\n",
    "root = C.load_model(modelPath)\n",
    "\n",
    "for i_episode in range(num_episodes):\n",
    "    print(i_episode)\n",
    "    observation = env.reset()  # reset environment for new episode\n",
    "    done = False\n",
    "    while not done: \n",
    "        if not 'TEST_DEVICE' in os.environ:\n",
    "            env.render()\n",
    "        action = np.argmax(root.eval([observation.astype(np.float32)]))\n",
    "        observation, reward, done, info  = env.step(action)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Policy gradient (PG)\n",
    "**Goal:**\n",
    "$$\n",
    "\\text{maximize } E [R | \\pi_\\theta]\n",
    "$$\n",
    "\n",
    "**Approach:**\n",
    "1. Collect experience (sample a bunch of trajectories through $(s,a)$ space)\n",
    "2. Update the policy so that _good_ experiences become more probable\n",
    "\n",
    "**Difference to DQN:**\n",
    " * we don't consider single $(s,a,r,s')$ transitions, but rather use whole episodes for the gradient updates\n",
    " * our parameters directly model the policy (output is an action probability), whereas in DQN they model the value function (output is raw score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rewards\n",
    "\n",
    "Remember, we get +1 reward for every time step, in which we still were in the game.\n",
    "\n",
    "The problem: we normally do not know, which action led to a continuation of the game, and which was actually a bad one. Our simple heuristic: actions in the beginning of the episode are good, and those towards the end are likely bad (they led to losing the game after all)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def discount_rewards(r, gamma=0.999):\n",
    "    \"\"\"Take 1D float array of rewards and compute discounted reward \"\"\"\n",
    "    discounted_r = np.zeros_like(r)\n",
    "    running_add = 0\n",
    "    for t in reversed(range(0, r.size)):\n",
    "        running_add = running_add * gamma + r[t]\n",
    "        discounted_r[t] = running_add\n",
    "    return discounted_r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x21b79d49470>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATcAAACSCAYAAAAghI3bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADMVJREFUeJzt3X9M1PUfB/DnXSc/BATiC6tZAbOxfrDV1Bp9He5ykqdr\nLQRXciAu1gZkI6I6+aGETH60trKGeeTKdbCxNmnZP1JkWvJtJS2YtNHmwCWnIqAEQsj9eH//cCKZ\ninzu84bj3fOxuXFyfF6vO70nn8993q/7GIQQAkREijHOdwNERDIw3IhISQw3IlISw42IlMRwIyIl\nMdyISEl3FG6dnZ3IysoCAPzxxx/IyMhAZmYmKioqpDZHRKTVjOG2f/9+lJWVweVyAQCqq6vx+uuv\no6GhAV6vF62trdKbJCKarRnDLTY2FnV1dVO3f/vtN6xcuRIAsHr1avz444/yuiMi0sg00x1SUlLg\ndDqnbk8faAgJCcHo6Ohtf35iYgJdXV2Ijo7GXXfd5UOrRETXeTweDAwMIDExEUFBQf/4/ozhdiOj\n8frO3tjYGJYsWXLb+3d1dcFqtc62DBHRHWlsbJw6mpxu1uH2yCOP4MSJE3jiiSfw/fffIykp6bb3\nj46OnmogJiYGe5r/h0uX/5pt2RlFhgajYON//xa+RKSu8+fPw2q1TmXMjWYdbjabDTt27IDL5cKy\nZctgsVhue/9rh6L33HMP7r33XkwYgzHq1X9WP9gYjKVLl/LQl+hf5lav+TsKt6VLl6KpqQkAEBcX\nB4fDoV9nREQS8BiOiJTEcCMiJTHciEhJDDciUhLDjYiUxHAjIiUx3IhISbNexLuQCCHg9XqlbNto\nNMJgMEjZNhH5Tulw83q92OU4gsGRcV23+58li7Ezaw2nIYj8mNLhBgCDI+O4MKxvuBGR/+N7bkSk\nJIYbESmJ4UZESmK4EZGSGG5EpCRNZ0vdbjdsNhucTidMJhMqKysRHx+vd29ERJpp2nM7duwYvF4v\nmpqakJ+fj/fee0/vvoiIfKIp3OLi4uDxeCCEwOjoKBYtWqR3X0REPtF0WBoSEoK+vj5YLBYMDw/D\nbrfr3deCxHEvIv+hKdwOHDiA5ORkFBYWor+/H1u2bMFXX32FgIAAvftbUDjuReQ/NIVbeHg4TKar\nPxoWFga32y1tj2Wh4bgXkX/QFG7Z2dkoKSmB1WqF2+1GUVHRTa/4TEQ0XzSF2+LFi/H+++/r3QsR\nkW64iJeIlMRwIyIlMdyISEkMNyJSEsONiJTEcCMiJSl/DQWVcdyL6NYYbgsYx72Ibo3htsBx3Ivo\n5vieGxEpieFGREpiuBGRkhhuRKQkhhsRKUnz2dL6+nocOXIELpcLGRkZSEtL07MvIiKfaAq3n3/+\nGb/++iuampowPj6OTz75RO++iIh8oincjh8/joSEBOTn52NsbAxvvfWW3n0REflEU7hdunQJZ8+e\nhd1ux5kzZ5CXl4fDhw/r3Rv5GY570UKiKdwiIiKwbNkymEwmxMfHIzAwEBcvXsTdd9+td3/kRzju\nRQuJpnBbsWIFHA4Htm7div7+fkxMTCAyMlLv3sgPcdyLFgpN4WY2m9He3o709HQIIVBeXs5DCiLy\nK5qXgrzxxht69kFEpCsu4iUiJTHciEhJDDciUhLDjYiUxHAjIiUx3IhISbyGAvktjnuRLxhu5Lc4\n7kW+YLiRX+O4F2nF99yISEkMNyJSEsONiJTEcCMiJTHciEhJPoXb0NAQzGYzent79eqHiEgXmsPN\n7XajvLwcQUFBevZDRKQLzeFWW1uLzZs3IyYmRs9+iIh0oSncmpubERUVhVWrVkEIoXdPRPNCCAGP\nxyPlD18nc0/ThEJzczMMBgPa2trQ3d0Nm82Gjz76CFFRUXr3RzRnOO6lFk3h1tDQMPV1VlYWdu3a\nxWAjJXDcSx0+LwXhJysQkT/yeXD+s88+06MPIiJdcREvESmJ4UZESmK4EZGSGG5EpCSGGxEpieFG\nREriNRSI5gmv7iUXw41onnDcSy6GG9E84riXPHzPjYiUxHAjIiUx3IhISQw3IlISw42IlKTpbKnb\n7UZJSQmcTidcLhdyc3OxZs0avXsjItJMU7gdOnQIkZGReOedd/Dnn3/i+eefZ7gRkV/RFG7r16+H\nxWIBcHUhosnE5XJE5F80pVJwcDAA4PLlyygoKEBhYaGuTRGRvmSOegH+Oe6leZfr3Llz2LZtGzIz\nM7FhwwY9eyIincka9QL8d9xLU7gNDg4iJycHO3fuRFJSkt49EZEE/7ZRL01LQex2O0ZGRrB3715k\nZWVhy5YtmJyc1Ls3IiLNNO25lZaWorS0VO9eiIh0w0W8RKQkhhsRKYnhRkRKYrgRkZIYbkSkJIYb\nESmJ4UZESuLEOxHpbi5mWWfCcCMi3c3FLOtMGG5EJMV8z7LyPTciUhLDjYiUxHAjIiUx3IhISQw3\nIlKSprOlQgi8/fbb+P333xEQEIDdu3fj/vvv17s3IiLNNO25tba2YnJyEk1NTSgqKkJ1dbXefRER\n+URTuP3yyy9ITk4GADz22GPo6urStSkiIl9pOiy9fPkywsLCrm/EZILX673pSITH4wEAnD9/Hl6v\nF0HevxBmnNDY7q0FeQ1wOp1/60FWvZvVYj3W8+d6qrz2pte7cOECgOsZcyODEELMduM1NTV4/PHH\npy7MbDabcfTo0Zvet729HVardbYliIjuSGNjI1auXPmPv9e057Z8+XJ89913sFgs6OjoQEJCwi3v\nm5iYiMbGRkRHR/vddQ2JaOHyeDwYGBhAYmLiTb+vac9t+tlSAKiurkZ8fLxvnRIR6UhTuBER+Tsu\n4iUiJTHciEhJDDciUpJffljlfI13dXZ24t1334XD4ZBax+12o6SkBE6nEy6XC7m5uVizZuZPFtXK\n6/WirKwMvb29MBqNqKiowIMPPiitHgAMDQ0hLS0Nn3766ZycbNq4cSNCQ0MBAPfddx+qqqqk1aqv\nr8eRI0fgcrmQkZGBtLQ0abW++OILNDc3w2Aw4MqVK+ju7kZbW9vUY9Wb2+2GzWaD0+mEyWRCZWWl\n1H+/yclJFBcXo6+vD6GhoSgvL8cDDzygz8aFH/r666/F9u3bhRBCdHR0iLy8POk1P/74Y/Hss8+K\nF154QXqtgwcPiqqqKiGEEMPDw8JsNkut980334iSkhIhhBA//fST9OfT5XKJV155Raxbt0709PRI\nrSWEEFeuXBGpqanS6whx9fnLzc0VQggxNjYmPvzwwzmpK4QQFRUV4vPPP5dao7W1Vbz22mtCCCHa\n2trEq6++KrVeQ0OD2LFjhxBCiJ6eHvHSSy/ptm2/PCydj/Gu2NhY1NXVSa8DAOvXr0dBQQGAq3tV\nJpPcHei1a9eisrISAOB0OhEeHi61Xm1tLTZv3oyYmBipda7p7u7G+Pg4cnJysHXrVnR2dkqrdfz4\ncSQkJCA/Px95eXl4+umnpdWa7uTJkzh16hQ2bdoktU5cXBw8Hg+EEBgdHcWiRYuk1jt16hRWr14N\nAIiPj0dPT49u2/bLw9LZjHfpJSUlBU6nU9r2pwsODgZw9XEWFBSgsLBQek2j0Yjt27ejtbUVH3zw\ngbQ6zc3NiIqKwqpVq7Bv3z5pdaYLCgpCTk4ONm3ahNOnT+Pll19GS0uLlP8vly5dwtmzZ2G323Hm\nzBnk5eXh8OHDute5UX19PbZt2ya9TkhICPr6+mCxWDA8PAy73S613sMPP4yjR49i7dq16OjowIUL\nFyCEgMFg8HnbfrnnFhoairGxsanbsoNtPpw7dw7Z2dlITU3Fhg0b5qRmTU0NWlpaUFZWhokJ/Wf+\ngKvh1tbWhqysLHR3d8Nms2FoaEhKrWvi4uLw3HPPTX0dERGBgYEBKbUiIiKQnJwMk8mE+Ph4BAYG\n4uLFi1JqXTM6OorTp0/jySeflFoHAA4cOIDk5GS0tLTg0KFDsNlsmJyclFYvLS0NISEhsFqt+Pbb\nb/Hoo4/qEmyAn4bb8uXLcezYMQCYcbxLb2IO1jQPDg4iJycHb775JlJTU6XX+/LLL1FfXw8ACAwM\nhNFolPbLoqGhAQ6HAw6HAw899BBqa2sRFRUlpdY1Bw8eRE1NDQCgv78fY2NjiI6OllJrxYoV+OGH\nH6ZqTUxMIDIyUkqta06cOIGkpCSpNa4JDw+fOlkRFhYGt9st9fqjJ0+exFNPPYXGxkasW7dO1xOH\nfnlYmpKSgra2Nrz44osAMKefF6fXb43bsdvtGBkZwd69e1FXVweDwYD9+/cjICBASr1nnnkGxcXF\nyMzMhNvtRmlpqbRa083FcwkA6enpKC4uRkZGBoxGI6qqqqSFt9lsRnt7O9LT0yGEQHl5ufTH2dvb\nO2cfBpudnY2SkhJYrVa43W4UFRUhKChIWr3Y2Fjs2bMH+/btw5IlS7B7927dts3xKyJSkl8elhIR\n+YrhRkRKYrgRkZIYbkSkJIYbESmJ4UZESmK4EZGSGG5EpKT/A5cLN67q6saCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b79d29d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "discounted_epr = discount_rewards(np.ones(10))\n",
    "f, ax = plt.subplots(1, figsize=(5,2))\n",
    "sns.barplot(list(range(10)), discounted_epr, color=\"steelblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We normalize the rewards so that they tank below zero towards the end. gamma controls how late the rewards tank."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x21b79f31668>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAACSCAYAAAA0ERM6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEhdJREFUeJzt3WlMVOfbBvALOsOgc0DUakzUDoZKq+ISMUZrgHHh70aI\nCNOKOKCQGtziLgLWpQZB0ybaBiJWIw2aigpG0zRalZa0pFqKccNAoiCpVFvQWphRloHn/dAwr7hV\nz5yD4rl+n5h58Nw3ai5m5pznPm5CCAEiIg1yf9UNEBG9KgxAItIsBiARaRYDkIg0iwFIRJrFACQi\nzdLJ+UMOhwMpKSmoqalBS0sLEhMTMWnSJOd6YWEhsrKyoNPpEBkZCYvFoljDRERKkRWAJ06cQM+e\nPbFjxw78888/mDVrljMAHQ4HMjIyUFBQAIPBgOjoaEyePBm9evVStHEiIlfJegs8ffp0LF++HADQ\n1tYGne7/c/TGjRswmUyQJAl6vR6BgYEoKSlRplsiIgXJegXYrVs3AIDNZsPy5cuxcuVK55rNZoOX\nl5fzsdFoRENDw3OP19jYiKtXr6JPnz5466235LRERPSE1tZW1NbWIiAgAJ6enk+sywpAALh9+zaW\nLl2KefPmYcaMGc7nJUmCzWZzPrbb7fD29n7usa5evYqYmBi5rRARPdfBgwcxZsyYJ56XFYB1dXVI\nSEjAxo0bMW7cuA5rfn5+qK6uRn19PTw9PVFSUoKEhITnHq9Pnz7OJvv16yenJSKiJ9y5cwcxMTHO\njHmcrADMzs5GfX09srKykJmZCTc3N3z44Yd4+PAhLBYLkpOTER8fDyEELBYL+vbt+9zjtb/t7dev\nHwYMGCCnJSKiZ3rWR2uyAjA1NRWpqanPXDebzTCbzXIOTUTUaXghNBFpFgOQiDSLAUhEmsUAJCLN\nYgASkWYxAIlIs1wKwEuXLsFqtT7xfE5ODsLCwhAbG4vY2FjcvHnTlTJERKqQvRVu7969OH78OIxG\n4xNrZWVl2LFjB4YOHfrSxxVCoK2tTW5bz+Xu7g43NzdVjk1EXY/sADSZTMjMzMS6deueWCsrK0N2\ndjZqa2thNpuxcOHCFz5uW1sbPs0tRF39A7mtPdXb3t2x0TqJwxaIyEl2AIaGhqKmpuapazNnzkRM\nTAwkScKSJUtQVFSEkJCQFz52Xf0D/HVf2QAkInqcKidB4uLi4OPjA51Oh5CQEFy7dk2NMkRELnE5\nAIUQHR7bbDaEhYXh4cOHEELg3LlzGDZsmKtliIgUJ/stcLv2kwrffvutcxrMqlWrYLVaYTAYMH78\neAQHB7vcKBGR0lwKwP79++PQoUMAgLCwMOfz4eHhCA8Pd60zIiKV8UJoItIsBiARaRYDkIg0y+WT\nIF0dd54QaZfmA5A7T4i0S/MBCHDnCZFWqTINprCwEFFRUZgzZw6OHDniSgkiItUoPg3G4XAgIyMD\nBQUFMBgMiI6OxuTJk9GrVy+XmyUiUpLsV4Dt02Aed+PGDZhMJkiSBL1ej8DAQJSUlLjUJBGRGmQH\nYGho6FM/4LfZbPDy8nI+NhqNaGhokFuGiEg1il8HKEkSbDab87Hdboe3t7fSZYiIXKb4NBg/Pz9U\nV1ejvr4ezc3NKCkpwahRo1wtQ0SkOFWmwSQnJyM+Ph5CCFgsFvTt29flRomIlKbKNBiz2Qyz2exS\nY28q7jwhen3wQuhOxp0nRK8PBuArwJ0nRK8HToMhIs1iABKRZjEAiUizGIBEpFmyToIIIbB582ZU\nVFTAw8MDaWlpGDhwoHM9JycHR48edQ5A+PTTT+Hr66tIw0RESpEVgGfOnEFzczMOHTqES5cuIT09\nHVlZWc71srIy7NixA0OHDlWsUSIipckKwNLSUgQFBQEARo4ciatXr3ZYLysrQ3Z2Nmpra2E2m7Fw\n4ULXOyVZeOE10bPJCsDHJ77odDq0tbXB3f3fjxRnzpyJmJgYSJKEJUuWoKioCCEhIcp0TC+FF14T\nPZusAJQkCXa73fn40fADgLi4OEiSBAAICQnBtWvXGICvEC+8Jno6WWeBR48ejaKiIgDAxYsX4e/v\n71yz2WwICwvDw4cPIYTAuXPnMGzYMGW6JSJSkKxXgKGhoSguLsacOXMAAOnp6R2mwaxatQpWqxUG\ngwHjx49HcHCwok0TESlBVgC6ublhy5YtHZ4bNGiQ8+vw8HCEh4e71hkRkcp4ITQRaRYDkIg0iwFI\nRJrFACQizeJAVFKMmrtOAO48IeUxAEkxau06AbjzhNShyjSYwsJCZGVlQafTITIyEhaLRbGG6fXG\nXSfUlcj6DPDRaTCrV69Genq6c83hcCAjIwM5OTnIzc1FXl4e7t27p1jDRERKkRWAz5sGc+PGDZhM\nJkiSBL1ej8DAQJSUlCjTLRGRghSfBvP4mtFoRENDw0sd/23v7nLakn1M1lOunhq1nnXczj7p8ibV\ne9oJpTe93tMoPg1GkiTYbDbnmt1uh7e39wsf293dHRutk+S09ULHZj316qlZ62n1Ovuky5tS71kn\nlN70ek8jKwBHjx6NH374AdOmTXtiGoyfnx+qq6tRX18PT09PlJSUICEh4YWP7ebm1qln+liva9Zq\n19knXViva9d7nCrTYJKTkxEfHw8hBCwWC/r27ato00RESlBlGozZbIbZbHapMSIitfFCaOrSOvOk\nC715GIDUZXX2SRd68zAAqct6FSdd6M3CX3FEpFkMQCLSLAYgEWmWrM8Am5qasHbtWty9exeSJCEj\nIwM9e/bs8D1paWm4cOECjEYjACArK8t5r2AioteBrAD85ptv4O/vj6VLl+K7775DVlYWUlNTO3xP\nWVkZ9u3bBx8fH0UaJSJSmuxpMO33+g0ODsYvv/zSYV0IgerqamzcuBHR0dHIz893vVMiIoX95yvA\no0eP4uuvv+7w3Ntvv+18O2s0GjsMPwCABw8ewGq1YsGCBXA4HIiNjcXw4cM77BkmInrV/jMAo6Ki\nEBUV1eG5ZcuWOafB2O32DuOvAKBbt26wWq0wGAwwGAwYN24cysvLGYDU5XHnyZtF9jSYoqIiDB8+\nHEVFRRgzZkyH9aqqKqxcuRLHjx+Hw+FAaWkpZs+erUjDRK8Kd568eWQFYHR0NJKSkjB37lx4eHjg\n888/BwDk5OTAZDJh4sSJmDVrFiwWC/R6PSIiIuDn56do40SdjTtP3jyyAtDT0xO7du164vn58+c7\nv46Pj0d8fLzsxoiI1MbX3ESkWQxAItIsBiARaRYDkIg0iwFIRJrFgahEr7HOvoe01rgUgKdPn8bJ\nkyed1wE+6vDhw8jLy4Ner0diYiJvkkT0kjr7HtJaJDsA09LSUFxcjCFDhjyxVldXh9zcXBw7dgyN\njY2Ijo7GhAkToNfrXWqWSEt44bX6ZP8aGD16NDZv3vzUtcuXLyMwMBA6nQ6SJMHX1xcVFRVySxER\nqULWNJj09HRMnz4dv/7661P/jM1m6zAgoXv37mhoaHCxVSIiZcmaBvNfJEnqMCLLbrfD29v75bsj\nIlKRKp+EjhgxAqWlpWhubkZDQwMqKysxePBgNUoREcmm6GUwj06DsVqtmDt3LoQQWLVqFTw8PJQs\nRUTkMpcCcOzYsRg7dqzz8aPTYCwWCywWiyuHJyJSFS8GIiLN4k4QInLS2s4TBiARAdDmzhMGIBEB\n0ObOk9czlomIOgEDkIg0S7VpMGlpabhw4QKMRiMAICsry3kzdSKi14Eq02AAoKysDPv27YOPj4/s\n5oiI1KTKNBghBKqrq7Fx40ZER0cjPz9fbhkiItWoMg3mwYMHsFqtWLBgARwOB2JjYzF8+HD4+/sr\n0zURkQJUmQbTrVs3WK1WGAwGGAwGjBs3DuXl5QxAInqtqHIdYFVVFVauXInjx4/D4XCgtLQUs2fP\nVqMUEXVhr3rniWrTYGbNmgWLxQK9Xo+IiAj4+fkpWYqIurjXYeeJmxBCqNLBS7h16xYmT56Ms2fP\nYsCAAa+6HSJ6Q/xXtvBCaCLSLAYgEWkWA5CINIsBSESaxQAkIs1iABKRZsm6DtBms2HNmjWw2+1o\naWnB+vXrMWrUqA7fc/jwYeTl5UGv1yMxMRFms1mJfomIFCMrAPfv348PPvgAsbGxqKqqwurVq1FQ\nUOBcr6urQ25uLo4dO4bGxkZER0djwoQJ0Ov1ijVOROQqWQG4YMEC531+HQ4HDAZDh/XLly8jMDAQ\nOp0OkiTB19cXFRUVCAgIcL1jIiKFyJ4GExAQgNraWqxbtw6pqakd1m02G7y8vJyPu3fvjoaGhmfW\naG1tBQDcuXPnpZonInqe9kxpz5jHyZ4GU1FRgTVr1iApKQljxozpsCZJEmw2m/Ox3W6Ht7f3M2vU\n1tYCAGJiYv6rHSKil1ZbWwuTyfTE87LeAl+/fh0rVqzAzp078d577z2xPmLECOzcuRPNzc1oampC\nZWUlBg8e/MzjBQQE4ODBg+jTp4/m7kpFROppbW1FbW3tMz9+kzUMYfHixaioqED//v0hhIC3tzcy\nMzM7TIM5cuQI8vLyIITAokWLMGXKFJd/GCIiJb0W02CIiF4FXghNRJrFACQizWIAEpFmqXJPkM4g\nhMDmzZtRUVEBDw8PpKWlYeDAgarWvHTpEj777DPk5uaqWsfhcCAlJQU1NTVoaWlBYmIiJk1SZ3Q4\nALS1tWHDhg2oqqqCu7s7tmzZgnfffVe1eu3u3r2LyMhI7N+/H4MGDVK11uzZsyFJEgBgwIAB2LZt\nm6r19uzZg8LCQrS0tGDu3LmIjIxUrdaxY8dQUFAANzc3NDU1oby8HMXFxc6fV2kOhwNJSUmoqamB\nTqfD1q1bVf33a25uRnJyMm7dugVJkrBp0ya88847yhxcdFHff/+9WL9+vRBCiIsXL4pFixapWu+r\nr74SYWFh4qOPPlK1jhBC5Ofni23btgkhhLh//74wm82q1jt9+rRISUkRQghx/vx51f8uhRCipaVF\nLFmyREydOlVUVlaqWqupqUlERESoWuNR58+fF4mJiUIIIex2u/jyyy87rfaWLVvE4cOHVa1x5swZ\nsWLFCiGEEMXFxWLZsmWq1jtw4ID45JNPhBBCVFZWivj4eMWO3WXfApeWliIoKAgAMHLkSFy9elXV\neiaTCZmZmarWaDd9+nQsX74cwL+vznQ6dV+oT5kyBVu3bgUA1NTUoEePHqrWA4Dt27cjOjoaffv2\nVb1WeXk5Hjx4gISEBMyfPx+XLl1Std7PP/8Mf39/LF68GIsWLcLEiRNVrdfuypUruH79OiwWi6p1\nfH190draCiEEGhoaVN/jf/36dQQHBwMABg0ahMrKSsWO3WXfAj++3U6n06Gtre2F7wb1skJDQ1FT\nU6PKsR/XrVs3AP/+jMuXL8fKlStVr+nu7o7169fjzJkz+OKLL1StVVBQgN69e2PChAnYvXu3qrUA\nwNPTEwkJCbBYLLh58yY+/vhjnDp1SrX/K3///Tf++OMPZGdn4/fff8eiRYtw8uRJVWo9as+ePVi6\ndKnqdYxGI27duoVp06bh/v37yM7OVrXekCFD8OOPP2LKlCm4ePEi/vrrLwgh4Obm5vKxu+wrQEmS\nYLfbnY/VDL9X4fbt24iLi0NERARmzJjRKTUzMjJw6tQpbNiwAY2NjarVKSgoQHFxMaxWK8rLy5GU\nlIS7d++qVs/X1xfh4eHOr318fJzbL9Xg4+ODoKAg6HQ6DBo0CAaDAffu3VOtHgA0NDTg5s2bGDt2\nrKp1gH9vfxsUFIRTp07hxIkTSEpKQnNzs2r1IiMjYTQaERMTg7Nnz2LYsGGKhB/QhQNw9OjRKCoq\nAgBcvHgR/v7+nVJXdMJ143V1dUhISMDatWsRERGher3jx49jz549AACDwQB3d3dVf5kcOHAAubm5\nyM3Nxfvvv4/t27ejd+/eqtXLz89HRkYGAODPP/+E3W5Hnz59VKsXGBiIn376yVmvsbERPXv2VK0e\nAJSUlGDcuHGq1mjXo0cP5wkWLy8vOBwOtLW1qVbvypUrGD9+PA4ePIipU6cqerKzy74FDg0NRXFx\nMebMmQPg3wk1nUGp3zzPk52djfr6emRlZSEzMxNubm7Yu3evcwSZ0v73v/8hOTkZ8+bNg8PhQGpq\nqmq1HtcZf59RUVFITk7G3Llz4e7ujm3btqka8GazGb/99huioqIghMCmTZtU/zmrqqpUvwqiXVxc\nHFJSUhATEwOHw4HVq1fD09NTtXomkwm7du3C7t274e3tjbS0NMWOza1wRKRZXfYtMBGRqxiARKRZ\nDEAi0iwGIBFpFgOQiDSLAUhEmsUAJCLNYgASkWb9Hxw/+fqDvP9NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b79d58be0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "discounted_epr_cent = discounted_epr - np.mean(discounted_epr)\n",
    "discounted_epr_norm = discounted_epr_cent/np.std(discounted_epr_cent)\n",
    "f, ax = plt.subplots(1, figsize=(5,2))\n",
    "sns.barplot(list(range(10)), discounted_epr_norm, color=\"steelblue\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x21b7f82d048>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUAAAADKCAYAAADUzJmgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGutJREFUeJzt3XtQVdf5PvDnHA83OSJqsLZqgDF1otIkA46jteiJSrxR\nGgI0XEQMTFKIOqhoEDAqcUB0TKaawlfQVhJk6hVHJ9NIVCqT8KuW0IEKDk4VdRQ1QRLLRbkczvr9\n4XAKohj32QuB/Xz+kr3JehdqHvdlrffohBACREQapH/eEyAiel4YgESkWQxAItIsBiARaRYDkIg0\niwFIRJplUPIfmc1mJCcno7a2Fu3t7YiNjcWcOXOs54uKipCVlQWDwYCgoCCEhISoNmEiIrUoCsAT\nJ05gxIgR2L59O/773//izTfftAag2WxGRkYGCgoK4ODggLCwMMydOxcjR45UdeJERLZSdAu8cOFC\nxMfHAwAsFgsMhv/l6JUrV+Du7g6j0Qg7Ozv4+PigtLRUndkSEalI0RWgk5MTAKCpqQnx8fFYvXq1\n9VxTUxOGDRtm/drZ2RmNjY29jtfS0oLKykq4ublhyJAhSqZERNRDR0cH6urq4OXlBUdHxx7nFQUg\nANy+fRsrVqzAkiVLsGjRIutxo9GIpqYm69fNzc1wcXHpdazKykpEREQonQoRUa/y8/MxderUHscV\nBeDdu3cRExODjRs3Yvr06d3OTZgwAdevX0dDQwMcHR1RWlqKmJiYXsdzc3OzTnL06NHY/PkZ1Dc8\nUDK1Jxrl4oTNS+dCr+9+12+xWLCz4P/hxyZ1640wOiH+rV9rqp4QAjK3lut0Ouh0Omnj0+Bz584d\nREREWDPmUYoCMDs7Gw0NDcjKykJmZiZ0Oh1+//vf48GDBwgJCUFSUhKio6MhhEBISAhGjx7d63id\nt71jxozBz3/+czi5jIS95b6SqT2Rk8tQjB07tsctdkdHB1r0Tmi0qPs/rpPeSXP1Ojo68FFeEe42\nqPtnBwAvuAzFxsg53eoJIWCxWFSv1Umv1zNwB4knPVpTFIApKSlISUl54nmTyQSTyaRkaBrg7jbc\nx/f31A/Ax7FYLH0auDT4KH4GSNQf9GXg8opz8GEAEv1EvOIcfBiARM+gL684ST7uBSYizWIAEpFm\n8RaYqJ+S+dKFL1weYgAS9VOyXrrwhcv/MACJ+jG+dJGLzwCJSLMYgESkWQxAItIsmwKwoqICkZGR\nPY7n5ubC398fS5cuxdKlS3Ht2jVbyhARSaH4JcjevXtx/PhxODs79zhXVVWF7du3Y/LkyTZNjohI\nJsVXgO7u7sjMzHzsuaqqKmRnZyM8PBw5OTmKJ0dEJJPiK0A/Pz/U1tY+9tzixYsREREBo9GI5cuX\no7i4GLNnz1Y8SSKST4sLr6WsA4yKioLRaAQAzJ49GxcvXmQAEvVzWlx4bXMAPtoCvampCf7+/vjy\nyy/h6OiIc+fOITg42NYyRNQHtLbw2uYA7Lys/eKLL6wt8desWYPIyEg4ODhgxowZmDVrls0TJSJS\nm00BOHbsWBw4cAAA4O/vbz0eEBCAgIAA22ZGRCQZF0ITkWYxAIlIsxiARKRZDEAi0iwGIBFpFgOQ\niDSLAUhEmsUAJCLNYgASkWYxAIlIsxiARKRZUlriFxUVITg4GKGhoTh8+LAtJYiIpFG9Jb7ZbEZG\nRgYKCgrg4OCAsLAwzJ07FyNHjrR5skREalK9Jf6VK1fg7u4Oo9EIOzs7+Pj4oLS01KZJEhHJoHpL\n/KamJgwbNsz6tbOzMxobG5WWIaJBqj+04Fe9Jb7RaERTU5P16+bmZri4uKhdhogGuP7Qgl/1lvgT\nJkzA9evX0dDQAEdHR5SWliImJsbWMkQ0CD3vFvxSWuInJSUhOjoaQgiEhIRg9OjRNk+UiEhtUlri\nm0wmmEwmmyZGRCQbF0ITkWYxAIlIsxiARKRZDEAi0iwGIBFpFgOQiDSLAUhEmsUAJCLNYgASkWYx\nAIlIsxiARKRZivYCCyGwefNmXLp0Cfb29khLS8P48eOt53Nzc3HkyBFrF+iPPvoIHh4eqkyYiEgt\nigLw9OnTaGtrw4EDB1BRUYGtW7ciKyvLer6qqgrbt2/H5MmTVZsoEZHaFAVgWVkZfH19AQCvvvoq\nKisru52vqqpCdnY26urqYDKZ8N5779k+UyIilSl6Bvho23uDwdCttfXixYuRmpqKzz//HGVlZSgu\nLrZ9pkREKlMUgEajEc3NzdavLRYL9Pr/DRUVFQVXV1cYDAbMnj0bFy9etH2mREQqUxSA3t7e1qu6\n8vJyTJw40XquqakJ/v7+ePDgAYQQOHfuHKZMmaLObImIVKToGaCfnx9KSkoQGhoKANi6dWu3lvhr\n1qxBZGQkHBwcMGPGDMyaNUvVSRMRqUFRAOp0OqSmpnY75unpaf11QEAAAgICbJsZEZFkXAhNRJrF\nACQizWIAEpFmMQCJSLMYgESkWQxAItIsBiARaRYDkIg0iwFIRJrFACQizWIAEpFmKQpAIQQ2bdqE\n0NBQLF26FDdu3Oh2vqioCMHBwQgNDcXhw4dVmSgRkdoUBWDXlvgJCQnYunWr9ZzZbEZGRgZyc3OR\nl5eHgwcP4ocfflBtwkREalEUgL21xL9y5Qrc3d1hNBphZ2cHHx8flJaWqjNbIiIVKWqH9aSW+Hq9\nvsc5Z2dnNDY29jpeR0cHAODOnTuwWCxwtDzAMH2Lkqk9kaNFh9ra2m6dqwGwnor1ZNVivcH3d6Wv\n6t25cwfA/zLmUYoCsLeW+EajEU1NTdZzzc3NcHFx6XW8uro6AEBERISS6fxkX/yf1OFZj/UGTL3B\n/LM9rl5dXR3c3d17fJ+iAPT29sbf//53LFiwoEdL/AkTJuD69etoaGiAo6MjSktLERMT0+t4Xl5e\nyM/Ph5ubG4YMGaJkSkREPXR0dKCurg5eXl6PPa8TQohnHbTrB6MDD1viV1VVWVvinz17Fn/6058g\nhEBwcDDCwsJs+ymIiCRQFIBERIMBF0ITkWYxAIlIsxiARKRZDEAi0ixFy2D6g65vou3t7ZGWlobx\n48dLrVlRUYEdO3YgLy9Pah2z2Yzk5GTU1taivb0dsbGxmDNnjrR6FosFGzZswNWrV6HX65GamoqX\nXnpJWr1O9fX1CAoKwr59+7p9rrQMb731FoxGIwBg3LhxSE9Pl1ovJycHRUVFaG9vR3h4OIKCgqTV\nOnbsGAoKCqDT6dDa2orq6mqUlJRYf161mc1mJCYmora2FgaDAVu2bJH659fW1oakpCTcvHkTRqMR\nmzZtwosvvqjO4GKA+uqrr8T69euFEEKUl5eLuLg4qfX27Nkj/P39xdtvvy21jhBCHD16VKSnpwsh\nhLh3754wmUxS6506dUokJycLIYQ4f/689N9LIYRob28Xy5cvF/Pnzxc1NTVSa7W2torAwECpNbo6\nf/68iI2NFUII0dzcLD799NM+q52amioOHToktcbp06fFqlWrhBBClJSUiJUrV0qtt3//fvHhhx8K\nIYSoqakR0dHRqo09YG+Be9uPLIO7uzsyMzOl1ui0cOFCxMfHA3h4dWYwyL1QnzdvHrZs2QIAqK2t\nxfDhw6XWA4Bt27YhLCwMo0ePll6ruroa9+/fR0xMDJYtW4aKigqp9b755htMnDgR77//PuLi4vD6\n669LrdfpwoULuHz5MkJCQqTW8fDwQEdHB4QQaGxshJ2dndR6ly9fxqxZswAAnp6eqKmpUW3sAXsL\n3Nt+ZBn8/PxQW1srZexHOTk5AXj4M8bHx2P16tXSa+r1eqxfvx6nT5/Grl27pNYqKCjAqFGjMHPm\nTOzevVtqLQBwdHRETEwMQkJCcO3aNbz77rsoLCyU9nflxx9/xK1bt5CdnY0bN24gLi4OJ0+elFKr\nq5ycHKxYsUJ6HWdnZ9y8eRMLFizAvXv3kJ2dLbXepEmTcPbsWcybNw/l5eX4/vvvIYSATqezeWyb\n/gZUVFQgMjKyx/G+6AfY237kweD27duIiopCYGAgFi1a1Cc1MzIyUFhYiA0bNqClRf0N/50KCgpQ\nUlKCyMhIVFdXIzExEfX19dLqeXh4ICAgwPprV1dX6/5zGVxdXeHr6wuDwQBPT084ODhIbwnX2NiI\na9euYdq0aVLrAEBubi58fX1RWFiIEydOIDExEW1tbdLqBQUFwdnZGREREThz5gymTJmiSvgBNgTg\n3r17sWHDBrS3t3c73lf9AL29vVFcXAwAPfYjyyT6YOPM3bt3ERMTg3Xr1iEwMFB6vePHjyMnJwcA\n4ODgAL1eL/Ufk/379yMvLw95eXl4+eWXsW3bNowaNUpavaNHjyIjIwMA8N1336G5uRlubm7S6vn4\n+ODrr7+21mtpacGIESOk1QOA0tJSTJ8+XWqNTsOHD7e+YBk2bBjMZjMsFou0ehcuXMCMGTOQn5+P\n+fPnq/qyU/EtcOczsQ8++KDb8a79AAFY+wHOnz/ftpk+ws/PDyUlJQgNDQWAbk1ZZVLrX57eZGdn\no6GhAVlZWcjMzIROp8PevXthb28vpd4bb7yBpKQkLFmyBGazGSkpKdJqPaovfj+Dg4ORlJSE8PBw\n6PV6pKenSw14k8mEb7/9FsHBwdbu6bJ/zqtXr0pfBdEpKioKycnJiIiIgNlsRkJCAhwdHaXVc3d3\nx86dO7F79264uLggLS1NtbFt2gtcW1uLhIQEHDhwwHqsrKwM+fn5+OSTTwAAu3btwi9+8QsEBwc/\ncZyWlhZUVlayGwwRqaprN5jHhbTqL0GU9AOsrKyU3guQiLQrPz8fU6dO7XHc5gB89AJSST/Azucx\n+fn5GDNmjK1TIiIC8LDLfERExBOf+docgJ3PNr744gtrP8CkpCRER0dDCIGQkJCnrvXqvO0dM2YM\nxo0bZ+uUiIi6edKjNZsCcOzYsdbnf/7+/tbjJpMJJpPJlqGJiKQbPAvniIiekaIrQPGURgS5ubk4\ncuQIRo4cCQD46KOP4OHh8ZPHlrWmSK/X98myCyIaGBQFYNcPRq+oqMDWrVuRlZVlPV9VVYXt27dj\n8uTJzzy2xWLBR3lFuNtwX8nUnugFl6HYGDmnx7OAvg7cwV6PaCBRFIBPa0RQVVWF7Oxs1NXVwWQy\n4b333num8e823Mf399QNwCfp68AdzPVkhi3AwCX1qf7B6ACwePFiREREwGg0Yvny5SguLsbs2bPV\nmbEEfRm4g7merLAFnhzwRLZQ/YPRgYdbZTq3ws2ePRsXL17s1wFI6unLcOcVJ9lK9Q9Gb2pqgr+/\nP7788ks4Ojri3LlzvW6DI1KKV5xkK0UB+LhGBF0XQq9ZswaRkZFwcHDAjBkzrM0MidTW148TaHBR\nFIA6nQ6pqandjnX9TICAgABr/zUiov5qwHaEJuprfOY4+DAAiX4iPnMcfKTsBCkqKkJWVhYMBgOC\ngoKkf0gLUV/hM8fBRdFe4K47QRISErp1Y+6rlvhERLZSFIC97QTp2hLfzs7O2hKfiKi/UX0nyKPn\nnJ2d0djY+Ezjv+AyVMm0FI/JeurVk1FLq/UG+77x/rBPXfWdIEpa4nel1+uxMXKOkmn9pLFZT149\nmbW0WG8w7xt/HvUeR/WdIEpa4nel0+n69E0Y6w3MWlqoBwzefePPq96jpOwEedaW+ET0eH39uERr\npOwEYUt8Itv19eMSLeJCaKJ+6nnccmuNogBsbW3FunXrUF9fD6PRiIyMDIwYMaLb96SlpeFf//oX\nnJ2dAQBZWVnWFllERP2BogD861//iokTJ2LFihX429/+hqysLKSkpHT7nqqqKvz5z3+Gq6urKhMl\nIlKb4oXQnS2uZs2ahX/84x/dzgshcP36dWzcuBFhYWE4evSo7TMlIlLZU68Ajxw5gs8++6zbsRde\neMF6O+vs7Nxt3R8A3L9/H5GRkXjnnXdgNpuxdOlS/OpXv+q2XIaI6Hl7agAGBwf36Oi8cuVK60Lo\n5ubmbjs/AMDJycnaENXBwQHTp09HdXU1A5CI+hVFt8De3t4oLi4GABQXF2Pq1Kndzl+9ehVhYWEQ\nQqC9vR1lZWWYMmWK7bMlIlKRopcgYWFhSExMRHh4OOzt7fHxxx8DePiB6O7u7nj99dfx5ptvIiQk\nBHZ2dggMDMSECRNUnTgRka0UBaCjoyN27tzZ4/iyZcusv46OjkZ0dLTiiRERycbl4ESkWQxAItIs\nmwLw1KlTSEhIeOy5Q4cOISgoCKGhoTh79qwtZYiIpFC8FzgtLQ0lJSWYNGlSj3N3795FXl4ejh07\nhpaWFoSFhWHmzJmws7OzabJERGpSHIDe3t7w8/PDwYMHe5z797//DR8fHxgMBhiNRnh4eODSpUvw\n8vKyabJEJJfW2m8p2gmydetWLFy4EP/85z8f+9882hZ/6NChz9wWn4j6lhbbbynaCfI0trbFJ6K+\np8X2W1Ji+ZVXXkFZWRna2trQ2NiImpoa/PKXv5RRiohIMVUbonbdCRIZGYnw8HAIIbBmzRrY29ur\nWYqIyGY2BeC0adMwbdo069ddd4KEhIQgJCTEluGJiKTqn08miYj6gE1XgKdOncLJkyetzRC6Ykt8\nIurvpCyEBtgSn4j6P8W3wN7e3ti8efNjz7ElPhENBFIWQrMlPhENBFIWQrMlPhENBFLeArMlPhEN\nBNIWQrMlPhH1d9IWQrMlPhH1d1wITUSaxQAkIs1SdAvc1NSEtWvXorm5Ge3t7Vi/fj1ee+21bt9z\n6NAhHDx4EHZ2doiNjYXJZFJjvkREqlEUgPv27cOvf/1rLF26FFevXkVCQgIKCgqs59kSn4gGAkUB\n+M4771jbW5nNZjg4OHQ7z5b4RDQQKN4J4uXlhbq6OnzwwQdISUnpdp4t8YloIFC8E+TSpUtYu3Yt\nEhMTMXXq1G7n2BKfiAYCRW+BL1++jFWrVmHHjh34zW9+0+M8W+IT0UCg6BngJ598gra2NqSlpUEI\nARcXF2RmZrIlPhENKIoCMCsr67HH2RKfiAYSLoQmIs2SthCaLfGJqL+TshAaYEt8Iur/pCyE7toS\nv66uDsHBwQgKCrJ9tkREKpKyEJot8YloIJCyEJot8YloIJCyEJot8YloIJC2EJot8YnoaV5wGfpc\nx9QJIYTqM3hGN2/exNy5c3HmzBmMGzfueU+HiPqAEAIWi0XK2Hq9Hjqd7qnZouqHIhER/VQ6nQ5D\nhgx5rnPgThAi0ixFV4APHjxAQkICGhoaYG9vj4yMDIwePbrb97AlPhH1d4quAA8dOgQvLy/s378f\nv/3tb7Fnz55u5ztb4h88eBB79+7Fxx9/jPb2dlUmTESkFkVXgFFRUeh8d3Lr1i0MHz6823m2xCei\ngcCmnSBRUVH4z3/+g7/85S/dzj9rS/yOjg4AwJ07d55p8kREvenMlM6MeZTinSAA8Nlnn6GmpgZ/\n+MMfcOrUKevxZ22JX1dXBwCIiIh42nSIiJ5ZXV0d3N3dexxXdAuck5ODn/3sZ/jd736HoUOH9niV\n/corr+CPf/wj2tra0Nra+tSW+F5eXsjPz4ebm9tzfy1ORINHR0cH6urqnvj4TdFC6Pr6eiQmJqK1\ntRVCCKxduxavvfZat50ghw8fxsGDByGEQFxcHObNm2fzD0NEpKZ+sROEiOh54EJoItIsBiARaRYD\nkIg0a8A2QxBCYPPmzbh06RLs7e2RlpaG8ePHS61ZUVGBHTt2IC8vT2ods9mM5ORk1NbWor29HbGx\nsZgzZ460ehaLBRs2bMDVq1eh1+uRmpqKl156SVq9TvX19QgKCsK+ffvg6ekptdZbb71l/VCucePG\nIT09XWq9nJwcFBUVob29HeHh4VI/EuLYsWMoKCiATqdDa2srqqurUVJSIu1DyMxmMxITE1FbWwuD\nwYAtW7ZI/fNra2tDUlISbt68CaPRiE2bNuHFF19UZ3AxQH311Vdi/fr1QgghysvLRVxcnNR6e/bs\nEf7+/uLtt9+WWkcIIY4ePSrS09OFEELcu3dPmEwmqfVOnTolkpOThRBCnD9/XvrvpRBCtLe3i+XL\nl4v58+eLmpoaqbVaW1tFYGCg1BpdnT9/XsTGxgohhGhubhaffvppn9VOTU0Vhw4dklrj9OnTYtWq\nVUIIIUpKSsTKlSul1tu/f7/48MMPhRBC1NTUiOjoaNXGHrC3wGVlZfD19QUAvPrqq6isrJRaz93d\nHZmZmVJrdFq4cCHi4+MBPLw6MxjkXqjPmzcPW7ZsAQDU1tb22Noow7Zt2xAWFtajiYYM1dXVuH//\nPmJiYrBs2TJUVFRIrffNN99g4sSJeP/99xEXF4fXX39dar1OFy5cwOXLlxESEiK1joeHBzo6OiCE\nQGNjI+zs7KTWu3z5MmbNmgUA8PT0RE1NjWpjD9hb4Ee32xkMBlgsFuj1cjLdz88PtbW1UsZ+lJOT\nE4CHP2N8fDxWr14tvaZer8f69etx+vRp7Nq1S2qtgoICjBo1CjNnzsTu3bul1gIAR0dHxMTEICQk\nBNeuXcO7776LwsJCaX9XfvzxR9y6dQvZ2dm4ceMG4uLicPLkSSm1usrJycGKFSuk13F2dsbNmzex\nYMEC3Lt3D9nZ2VLrTZo0CWfPnsW8efNQXl6O77//HkII6HQ6m8cesFeARqMRzc3N1q9lht/zcPv2\nbURFRSEwMBCLFi3qk5oZGRkoLCzEhg0b0NLSIq1OQUEBSkpKEBkZierqaiQmJqK+vl5aPQ8PDwQE\nBFh/7erqat1+KYOrqyt8fX1hMBjg6ekJBwcH/PDDD9LqAUBjYyOuXbuGadOmSa0DALm5ufD19UVh\nYSFOnDiBxMREtLW1SasXFBQEZ2dnRERE4MyZM5gyZYoq4QcM4AD09vZGcXExAKC8vLzPPnFO9MG6\n8bt37yImJgbr1q1DYGCg9HrHjx9HTk4OAMDBwQF6vV7qPyb79+9HXl4e8vLy8PLLL2Pbtm0YNWqU\ntHpHjx5FRkYGAOC7775Dc3Mz3NzcpNXz8fHB119/ba3X0tKCESNGSKsHAKWlpZg+fbrUGp2GDx9u\nfcEybNgwmM1maa3tgYe39jNmzEB+fj7mz5+v6svOAXsL7Ofnh5KSEoSGhgJ42KGmL6j1L09vsrOz\n0dDQgKysLGRmZkKn02Hv3r3WD6NX2xtvvIGkpCQsWbIEZrMZKSkp0mo9qi9+P4ODg5GUlITw8HDo\n9Xqkp6dLDXiTyYRvv/0WwcHBEEJg06ZN0n/Oq1evSl8F0SkqKgrJycmIiIiA2WxGQkICHB0dpdVz\nd3fHzp07sXv3bri4uCAtLU21sbkVjog0a8DeAhMR2YoBSESaxQAkIs1iABKRZjEAiUizGIBEpFkM\nQCLSLAYgEWnW/wfyBumzL4N5TQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x21b7f1a2320>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "discounted_epr = discount_rewards(np.ones(10), gamma=0.5)\n",
    "discounted_epr_cent = discounted_epr - np.mean(discounted_epr)\n",
    "discounted_epr_norm = discounted_epr_cent/np.std(discounted_epr_cent)\n",
    "f, ax = plt.subplots(2, figsize=(5,3))\n",
    "sns.barplot(list(range(10)), discounted_epr, color=\"steelblue\", ax=ax[0])\n",
    "sns.barplot(list(range(10)), discounted_epr_norm, color=\"steelblue\", ax=ax[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model: Policy Gradient\n",
    "\n",
    "$$\n",
    "l_1 = relu( x W_1 + b_1) \\\\\n",
    "l_2 = l_1 W_2 + b_2 \\\\\n",
    "\\pi(a|s) = sigmoid(l_2)\n",
    "$$\n",
    "\n",
    "Note: in policy gradient approach, the output of the dense layer is mapped into to a 0-1 range via the sigmoid function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "TOTAL_EPISODES = 2000 if isFast else 10000\n",
    "\n",
    "D = 4  # input dimensionality\n",
    "H = 10 # number of hidden layer neurons\n",
    "\n",
    "observations = C.sequence.input_variable(STATE_COUNT, np.float32, name=\"obs\")\n",
    "\n",
    "W1 = C.parameter(shape=(STATE_COUNT, H), init=C.glorot_uniform(), name=\"W1\")\n",
    "b1 = C.parameter(shape=H, name=\"b1\")\n",
    "layer1 = C.relu(C.times(observations, W1) + b1)\n",
    "\n",
    "W2 = C.parameter(shape=(H, ACTION_COUNT), init=C.glorot_uniform(), name=\"W2\")\n",
    "b2 = C.parameter(shape=ACTION_COUNT, name=\"b2\")\n",
    "score = C.times(layer1, W2) + b2\n",
    "# Until here it was similar to DQN\n",
    "\n",
    "probability = C.sigmoid(score, name=\"prob\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the PG model\n",
    "\n",
    "**Policy Search**: The optimal policy search can be carried out with either gradient free approaches or by computing gradients over the policy space ($\\pi_\\theta$) which is parameterized by $\\theta$. In this tutorial, we use the classic forward (`loss.forward`) and back (`loss.backward`) propagation of errors over the parameterized space $\\theta$. In this case, $\\theta = \\{W_1, b_1, W_2, b_2\\}$, our model parameters.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode: 20. Average reward for episode 21.800000.\n",
      "Episode: 40. Average reward for episode 23.450000.\n",
      "Episode: 60. Average reward for episode 20.650000.\n",
      "Episode: 80. Average reward for episode 23.900000.\n",
      "Episode: 100. Average reward for episode 17.750000.\n",
      "Episode: 120. Average reward for episode 27.550000.\n",
      "Episode: 140. Average reward for episode 20.300000.\n",
      "Episode: 160. Average reward for episode 20.450000.\n",
      "Episode: 180. Average reward for episode 14.700000.\n",
      "Episode: 200. Average reward for episode 18.900000.\n",
      "Episode: 220. Average reward for episode 18.350000.\n",
      "Episode: 240. Average reward for episode 20.050000.\n",
      "Episode: 260. Average reward for episode 19.600000.\n",
      "Episode: 280. Average reward for episode 22.150000.\n",
      "Episode: 300. Average reward for episode 16.000000.\n",
      "Episode: 320. Average reward for episode 21.300000.\n",
      "Episode: 340. Average reward for episode 19.250000.\n",
      "Episode: 360. Average reward for episode 17.550000.\n",
      "Episode: 380. Average reward for episode 20.050000.\n",
      "Episode: 400. Average reward for episode 20.250000.\n",
      "Episode: 420. Average reward for episode 20.200000.\n",
      "Episode: 440. Average reward for episode 21.100000.\n",
      "Episode: 460. Average reward for episode 23.400000.\n",
      "Episode: 480. Average reward for episode 20.100000.\n",
      "Episode: 500. Average reward for episode 18.400000.\n",
      "Episode: 520. Average reward for episode 15.550000.\n",
      "Episode: 540. Average reward for episode 16.850000.\n",
      "Episode: 560. Average reward for episode 18.000000.\n",
      "Episode: 580. Average reward for episode 23.650000.\n",
      "Episode: 600. Average reward for episode 16.550000.\n",
      "Episode: 620. Average reward for episode 21.200000.\n",
      "Episode: 640. Average reward for episode 19.100000.\n",
      "Episode: 660. Average reward for episode 25.150000.\n",
      "Episode: 680. Average reward for episode 22.400000.\n",
      "Episode: 700. Average reward for episode 19.150000.\n",
      "Episode: 720. Average reward for episode 19.900000.\n",
      "Episode: 740. Average reward for episode 26.500000.\n",
      "Episode: 760. Average reward for episode 20.500000.\n",
      "Episode: 780. Average reward for episode 19.300000.\n",
      "Episode: 800. Average reward for episode 21.250000.\n",
      "Episode: 820. Average reward for episode 27.900000.\n",
      "Episode: 840. Average reward for episode 25.600000.\n",
      "Episode: 860. Average reward for episode 27.250000.\n",
      "Episode: 880. Average reward for episode 27.000000.\n",
      "Episode: 900. Average reward for episode 29.900000.\n",
      "Episode: 920. Average reward for episode 28.900000.\n",
      "Episode: 940. Average reward for episode 30.200000.\n",
      "Task solved in: 940 \n"
     ]
    }
   ],
   "source": [
    "input_y = C.sequence.input_variable(1, np.float32, name=\"input_y\")\n",
    "advantages = C.sequence.input_variable(1, np.float32, name=\"advt\")\n",
    "\n",
    "loss = -C.reduce_mean(C.log(C.square(input_y - probability) + 1e-4) * advantages, axis=0, name='loss')\n",
    "\n",
    "lr = 0.001\n",
    "lr_schedule = C.learning_rate_schedule(lr, C.UnitType.sample)\n",
    "sgd = C.sgd([W1, W2], lr_schedule)\n",
    "\n",
    "gradBuffer = dict((var.name, np.zeros(shape=var.shape)) for var in loss.parameters if var.name in ['W1', 'W2', 'b1', 'b2'])\n",
    "\n",
    "xs, hs, label, drs = [], [], [], []\n",
    "running_reward = None\n",
    "reward_sum = 0\n",
    "episode_number = 1\n",
    "\n",
    "observation = env.reset()\n",
    "\n",
    "while episode_number <= TOTAL_EPISODES:\n",
    "    x = np.reshape(observation, [1, STATE_COUNT]).astype(np.float32)\n",
    "\n",
    "    # Run the policy network and get an action to take.\n",
    "    prob = probability.eval(arguments={observations: x})[0][0][0]\n",
    "    action = 1 if np.random.uniform() < prob else 0\n",
    "\n",
    "    xs.append(x)  # observation\n",
    "    # grad that encourages the action that was taken to be taken\n",
    "\n",
    "    y = 1 if action == 0 else 0  # a \"fake label\"\n",
    "    label.append(y)\n",
    "\n",
    "    # step the environment and get new measurements\n",
    "    observation, reward, done, info = env.step(action)\n",
    "    reward_sum += float(reward)\n",
    "\n",
    "    # Record reward (has to be done after we call step() to get reward for previous action)\n",
    "    drs.append(float(reward))\n",
    "\n",
    "    if done:\n",
    "        # Stack together all inputs, hidden states, action gradients, and rewards for this episode\n",
    "        epx = np.vstack(xs)\n",
    "        epl = np.vstack(label).astype(np.float32)\n",
    "        epr = np.vstack(drs).astype(np.float32)\n",
    "        xs, label, drs = [], [], []  # reset array memory\n",
    "\n",
    "        # Compute the discounted reward backwards through time.\n",
    "        discounted_epr = discount_rewards(epr)\n",
    "        # Size the rewards to be unit normal (helps control the gradient estimator variance)\n",
    "        discounted_epr -= np.mean(discounted_epr)\n",
    "        discounted_epr /= np.std(discounted_epr)\n",
    "\n",
    "        # Forward pass\n",
    "        arguments = {observations: epx, input_y: epl, advantages: discounted_epr}\n",
    "        state, outputs_map = loss.forward(arguments, outputs=loss.outputs,\n",
    "                                          keep_for_backward=loss.outputs)\n",
    "\n",
    "        # Backward psas\n",
    "        root_gradients = {v: np.ones_like(o) for v, o in outputs_map.items()}\n",
    "        vargrads_map = loss.backward(state, root_gradients, variables=set([W1, W2]))\n",
    "\n",
    "        for var, grad in vargrads_map.items():\n",
    "            gradBuffer[var.name] += grad\n",
    "\n",
    "        # Wait for some batches to finish to reduce noise\n",
    "        if episode_number % BATCH_SIZE_BASELINE == 0:\n",
    "            grads = {W1: gradBuffer['W1'].astype(np.float32),\n",
    "                     W2: gradBuffer['W2'].astype(np.float32)}\n",
    "            updated = sgd.update(grads, BATCH_SIZE_BASELINE)\n",
    "\n",
    "            # reset the gradBuffer\n",
    "            gradBuffer = dict((var.name, np.zeros(shape=var.shape))\n",
    "                              for var in loss.parameters if var.name in ['W1', 'W2', 'b1', 'b2'])\n",
    "\n",
    "            print('Episode: %d. Average reward for episode %f.' % (episode_number, reward_sum / BATCH_SIZE_BASELINE))\n",
    "\n",
    "            if reward_sum / BATCH_SIZE_BASELINE > REWARD_TARGET:\n",
    "                print('Task solved in: %d ' % episode_number)\n",
    "                break\n",
    "\n",
    "            reward_sum = 0\n",
    "\n",
    "        observation = env.reset()  # reset env\n",
    "        episode_number += 1\n",
    "probability.save('pg.mod')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solutions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4, 10), (10,), (10, 2), (2,), (2,))"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observation = C.sequence.input_variable(STATE_COUNT, np.float32, name=\"s\")\n",
    "\n",
    "W1 = C.parameter(shape=(STATE_COUNT, H), init=C.glorot_uniform(), name=\"W1\")\n",
    "b1 = C.parameter(shape=H, name=\"b1\")\n",
    "layer1 = C.relu(C.times(observation, W1) + b1)\n",
    "\n",
    "W2 = C.parameter(shape=(H, ACTION_COUNT), init=C.glorot_uniform(), name=\"W2\")\n",
    "b2 = C.parameter(shape=ACTION_COUNT, name=\"b2\")\n",
    "model = C.times(layer1, W2) + b2\n",
    "W1.shape, b1.shape, W2.shape, b2.shape, model.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4, 10), (10,), (10, 2), (2,), (2,))"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Correspoding layers implementation - Preferred solution\n",
    "def create_model(input):\n",
    "    with C.layers.default_options(init=C.glorot_uniform()):\n",
    "        z = C.layers.Sequential([C.layers.Dense(H, name=\"layer1\"),\n",
    "                                 C.layers.Dense(ACTION_COUNT, name=\"layer2\")])\n",
    "        return z(input)\n",
    "    \n",
    "model = create_model(observation)\n",
    "\n",
    "model.layer1.W.shape, model.layer1.b.shape, model.layer2.W.shape, model.layer2.b.shape, model.shape"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}