https://github.com/GPflow/GPflow
Revision 7e717ccb69bf4a3e563f176b28ff2f516d65482c authored by st-- on 31 March 2020, 12:17:12 UTC, committed by GitHub on 31 March 2020, 12:17:12 UTC
This gives all `BayesianModel` subclasses a consistent interface both for optimization (MLE/MAP) and MCMC. Models are required to implement `maximum_log_likelihood_objective`, which is to be maximized for model training.

Optimization: The `_training_loss` method is defined as `- (maximum_log_likelihood_objective + log_prior_density)`. This is exposed by the InternalDataTrainingLossMixin and ExternalDataTrainingLossMixin classes.

For models that keep hold of the data internally, `training_loss` can directly be passed as a closure to an optimizer's `minimize`, for example:
```python
model = gpflow.models.GPR(data, ...)
gpflow.optimizers.Scipy().minimize(model.training_loss, model.trainable_variables)
```

If the model objective requires data to be passed in, a closure can be constructed on the fly using `model.training_loss_closure(data)`, which returns a no-argument closure:
```python
model = gpflow.models.SVGP(...)
gpflow.optimizers.Scipy().minimize(
    model.training_loss_closure(data), model.trainable_variables, ...
)
```

The training_loss_closure() method provided by both InternalDataTrainingLossMixin and ExternalDataTrainingLossMixin takes a boolean `compile` argument (default: True) that wraps the returned closure in tf.function(). Note that the return value should be cached in a variable if the minimize() step is run several times to avoid re-compilation in each step!

MCMC: The `log_posterior_density` method can be directly passed to the `SamplingHelper`. By default, `log_posterior_density` is implemented as `maximum_log_likelihood_objective + log_prior_density`. Models can override this if needed. Example:
```python
model = gpflow.models.GPMC(...)
hmc_helper = gpflow.optimizers.SamplingHelper(
    model.log_posterior_density, model.trainable_parameters
)
hmc = tfp.mcmc.HamiltonianMonteCarlo(
    target_log_prob_fn=hmc_helper.target_log_prob_fn, ...
)
```
In this case, the function that runs the MCMC chain should be wrapped in tf.function() (see MCMC notebook).
1 parent e61ee69
Raw File
Tip revision: 7e717ccb69bf4a3e563f176b28ff2f516d65482c authored by st-- on 31 March 2020, 12:17:12 UTC
refactor training objective methods (#1276)
Tip revision: 7e717cc
GLOSSARY.md
## Glossary

GPflow does not always follow standard Python naming conventions,
and instead tries to apply the notation in the relevant GP papers.\
The following is the convention we aim to use in the code.

---

<dl>
  <dt>GPR</dt>
  <dd>Gaussian process regression</dd>

  <dt>SVGP</dt>
  <dd>stochastic variational inference for Gaussian process models</dd>

  <dt>Shape constructions [..., A, B]</dt>
  <dd>the way of describing tensor shapes in docstrings and comments. Example: <i>[..., N, D, D]</i>, this is a tensor with an arbitrary number of leading dimensions indicated using the ellipsis sign, and the last two dimensions are equal</dd>

  <dt>X</dt>
  <dd>(and variations like Xnew) refers to input points; always of rank 2, e.g. shape <i>[N, D]</i>, even when <i>D=1</i></dd>

  <dt>Y</dt>
  <dd>(and variations like Ynew) refers to observed output values, potentially with multiple output dimensions; always of rank 2, e.g. shape <i>[N, P]</i>, even when <i>P=1</i></dd>

  <dt>Z</dt>
  <dd>refers to inducing points</dd>

  <dt>M</dt>
  <dd>stands for the number of inducing features (e.g. length of Z)</dd>

  <dt>N</dt>
  <dd>stands for the number of data or minibatch size in docstrings and shape constructions</dd>

  <dt>P</dt>
  <dd>stands for the number of output dimensions in docstrings and shape constructions</dd>

  <dt>D</dt>
  <dd>stands for the number of input dimensions in docstrings and shape constructions</dd>
</dl>
back to top