{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Settings\n",
    "GPflow has a config file, `gpflowrc` which allows the user to change the default behavious in GPflow. GPflow searches for the file in the following order:\n",
    "1. In the working directory\n",
    "2. In the user's home directory\n",
    "3. In the GPflow directory (revert to default)\n",
    "\n",
    "You can also make `gpflowrc` a hidden file, if you don't want it clutting your home directory, by renaming as `.gpflowrc`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, the configuration looks like this:\n",
    "```\n",
    "[verbosity]\n",
    "tf_compile_verb = False\n",
    "hmc_verb = True\n",
    "optimisation_verb = False\n",
    "\n",
    "[dtypes]\n",
    "float_type = float64\n",
    "int_type = int32\n",
    "\n",
    "[numerics]\n",
    "jitter_level = 1e-6\n",
    "\n",
    "[profiling]\n",
    "dump_timeline = False\n",
    "dump_tensorboard = False\n",
    "\n",
    "[session]\n",
    "intra_op_parallelism_threads = 0\n",
    "inter_op_parallelism_threads = 0\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing settings\n",
    "You can access the settings as `gpflow.settings`, and the different options are nested under the headings in the file. For example, to see how much jitter is added before attempting Cholesky decomposition:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1e-06\n"
     ]
    }
   ],
   "source": [
    "import gpflow\n",
    "print(gpflow.settings.jitter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modifying settings\n",
    "Settings can be modified for an entire session, or for a limited set of statements, using a Python context manager. It is recommeded to use the context manager, as this prevents the change of state unintentionally spilling into other parts of the program."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example\n",
    "By default, verbose compiling is switched off:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import numpy.random as rnd\n",
    "X = rnd.randn(100, 1)\n",
    "Y = np.sin(X) + np.sin(1.5*X) + 0.3 * rnd.randn(*X.shape)\n",
    "\n",
    "m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This can be switched on. First make a copy of the current settings using `get_settings()`, then modify and set using the context manager `temp_settings`. Finally, we see the compilation message showing up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Optimization terminated with:\n",
      "  Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n",
      "  Objective function value: 27.154970\n",
      "  Number of iterations: 14\n",
      "  Number of functions evaluations: 19\n"
     ]
    }
   ],
   "source": [
    "custom_config = gpflow.settings.get_settings()\n",
    "custom_config.verbosity.tf_compile_verb = True\n",
    "opt = gpflow.train.ScipyOptimizer()\n",
    "with gpflow.settings.temp_settings(custom_config), gpflow.session_manager.get_session().as_default():\n",
    "    m = gpflow.models.SGPR(X, Y, gpflow.kernels.RBF(1), Z=X.copy())\n",
    "    opt.minimize(m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Change TensorFlow session settings\n",
    "GPflow may create multiple tensorflow sessions for a single model; for example a separate session is created for each autoflow method. To control the session parameters change the [session] section of the settings. This section may contain any valid [TensorFlow ConfigProto](https://www.tensorflow.org/api_docs/python/tf/ConfigProto) setting. \n",
    "\n",
    "For instance to ensure all tensorflow graphs are run serially set\n",
    "```\n",
    "[session]\n",
    "intra_op_parallelism_threads = 1\n",
    "inter_op_parallelism_threads = 1\n",
    "```\n",
    "As per the TensorFlow documentation, a setting of 0 means the system picks an appropriate number of cores to use.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notes\n",
    "It's important to note that for some cases, a re-compilation of the model is necessary. For example, if we change the jitter level and optimise, the hyperparameters won't change unless we explicitly recompile the model. Additionally, state defined inside the context manager will be carried over to outside the context manager, until the next recompile.\n",
    "\n",
    "Essentially, to be safe, **if a model is to be used inside a context manager, everything should be done within the context manager**.\n",
    "\n",
    "### Example\n",
    "We first look at the kernel hyperparameters from the previous optimisation. Those inside the context manager will be the same, despite the drastically increased jitter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(1.231800598892151)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.kern.lengthscales.read_value()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Optimization terminated with:\n",
      "  Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n",
      "  Objective function value: 27.154962\n",
      "  Number of iterations: 8\n",
      "  Number of functions evaluations: 10\n",
      "1.2315556419374536\n"
     ]
    }
   ],
   "source": [
    "custom_config.numerics.jitter_level = 10e-0\n",
    "with gpflow.settings.temp_settings(custom_config):\n",
    "    opt.minimize(m)\n",
    "    print(m.kern.lengthscales.read_value())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the model is re-compiled, the modified jitter is taken into account in the TensorFlow graph, and the resulting hyperparameters are very different."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Optimization terminated with:\n",
      "  Message: b'CONVERGENCE: REL_REDUCTION_OF_F_<=_FACTR*EPSMCH'\n",
      "  Objective function value: 93.107377\n",
      "  Number of iterations: 167\n",
      "  Number of functions evaluations: 184\n",
      "1.0021331815790675\n"
     ]
    }
   ],
   "source": [
    "m.clear()\n",
    "with gpflow.settings.temp_settings(custom_config), gpflow.session_manager.get_session().as_default():\n",
    "    m.compile()\n",
    "    opt.minimize(m)\n",
    "    print(m.kern.lengthscales.read_value())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}