/*--------------------------------------------------------------------------
   Author: Thomas Nowotny
  
   Institute: Center for Computational Neuroscience and Robotics
              University of Sussex
	      Falmer, Brighton BN1 9QJ, UK 
  
   email to:  T.Nowotny@sussex.ac.uk
  
   initial version: 2010-02-07
  
--------------------------------------------------------------------------*/


//----------------------------------------------------------------------------
/*!  \page UserManual User Manual

\latexonly
\vspace{0cm}\mbox{}\vspace{0cm}
\endlatexonly

\section Contents
- \ref sIntro\n
- \subpage sectDefiningNetwork
- \subpage sectNeuronModels
- \subpage sectSynapseModels
- \subpage sect_postsyn
- \subpage sectCurrentSourceModels
- \subpage subsect34
- \subpage sectVariableInitialisation
- \subpage sectSparseConnectivityInitialisation

\section sIntro Introduction

GeNN is a software library for facilitating the simulation of neuronal
network models on NVIDIA CUDA enabled GPU hardware. It was designed
with computational neuroscience models in mind rather than artificial
neural networks. The main philosophy of GeNN is two-fold:

1. GeNN relies heavily on code generation to make it very flexible and
   to allow adjusting simulation code to the model of interest and the GPU
   hardware that is detected at compile time.
2. GeNN is lightweight in that it provides code for running models of
   neuronal networks on GPU hardware but it leaves it to the user to
   write a final simulation engine. It so allows maximal flexibility to
   the user who can use any of the provided code but can fully choose,
   inspect, extend or otherwise modify the generated code. They can also
   introduce their own optimisations and in particular control the data
   flow from and to the GPU in any desired granularity.

This manual gives an overview of how to use GeNN for a novice user and
tries to lead the user to more expert use later on. With that we jump
right in.
\n


-----
\link ReleaseNotes Previous\endlink | \link UserManual Top\endlink | \link sectDefiningNetwork Next\endlink
*/


//----------------------------------------------------------------------------
/*!  \page sectDefiningNetwork Defining a network model

A network model is defined by the user by providing the function 
\code{.cc}
void modelDefinition(ModelSpec &model) 
\endcode
in a separate file, such as `MyModel.cc`. In this function, the following
tasks must be completed:

1. The name of the model must be defined:
   \code{.cc}
   model.setName("MyModel");
   \endcode
2. Neuron populations (at least one) must be added (see \ref subsect11).
   The user may add as many neuron populations as they wish. If resources
   run out, there will not be a warning but GeNN will fail. However,
   before this breaking point is reached, GeNN will make all necessary
   efforts in terms of block size optimisation to accommodate the defined
   models. All populations must have a unique name.

3. Synapse populations (zero or more) can be added (see \ref subsect12). Again,
   the number of synaptic connection populations is unlimited other than
   by resources.

\section subsect11 Defining neuron populations

Neuron populations are added using the function
\code{.cc}
model.addNeuronPopulation<NeuronModel>(name, num, paramValues, varInitialisers);
\endcode
where the arguments are:
\arg `NeuronModel`:        Template argument specifying the type of neuron model
                           These should be derived off NeuronModels::Base and can
                           either be one of the standard models or user-defined
                           (see \ref sectNeuronModels).
\arg `const string &name`: Unique name of the neuron population
\arg `unsigned int size`: number of neurons in the population
\arg `NeuronModel::ParamValues paramValues`: Parameters of this neuron type
\arg `NeuronModel::VarValues varInitialisers`: Initial values or initialisation snippets for variables of this neuron type 

The user may add as many neuron populations as the model necessitates.
They must all have unique names. The possible values for the arguments,
predefined models and their parameters and initial values are detailed
\ref sectNeuronModels below.

\section subsect12 Defining synapse populations

Synapse populations are added with the function
\code{.cc}
model.addSynapsePopulation<WeightUpdateModel, PostsynapticModel>(name, mType, delay, preName, postName, 
                                                                 weightParamValues, weightVarValues, weightPreVarInitialisers, weightPostVarInitialisers,
                                                                 postsynapticParamValues, postsynapticVarValues, connectivityInitialiser);
\endcode
where the arguments are
\arg `WeightUpdateModel`:   Template parameter specifying the type of weight update model.
                            These should be derived off WeightUpdateModels::Base and can
                            either be one of the standard models or user-defined
                            (see \ref sectSynapseModels).
\arg `PostsynapticModel`:   Template parameter specifying the type of postsynaptic integration model.
                            These should be derived off PostsynapticModels::Base and can
                            either be one of the standard models or user-defined
                            (see \ref sect_postsyn).
\arg `const string &name`: The name of the synapse population
\arg `unsigned int mType`: How the synaptic matrix is stored. See \ref subsect34 for available options.
\arg `unsigned int delay`: Homogeneous (axonal) delay for synapse population (in terms of the simulation
     time step `DT`). 
\arg `const string preName`: Name of the (existing!) pre-synaptic
     neuron population.
\arg `const string postName`: Name of the (existing!) post-synaptic
     neuron population.
\arg `WeightUpdateModel::ParamValues weightParamValues`:    The parameter values (common to all synapses of 
                                                            the population) for the weight update model. 
\arg `WeightUpdateModel::VarValues weightVarInitialisers`:    Initial values or initialisation snippets for the weight
                                                        update model's state variables
\arg `WeightUpdateModel::PreVarValues weightPreVarInitialisers`:    Initial values or initialisation snippets for the weight
                                                        update model's presynaptic state variables
\arg `WeightUpdateModel::PostVarValues weightPostVarInitialisers`:    Initial values or initialisation snippets for the weight
                                                        update model's postsynaptic state variables
\arg `PostsynapticModel::ParamValues postsynapticParamValues`:  The parameter values (common to all 
                                                                postsynaptic neurons) for the postsynaptic model. 
\arg `PostsynapticModel::VarValues postsynapticVarInitialisers`:  Initial values or initialisation snippets for variables for the 
                                                            postsynaptic model's state variables
\arg `InitSparseConnectivitySnippet::Init connectivityInitialiser`: Optional argument, specifying the initialisation snippet for synapse population's sparse connectivity (see \ref sectSparseConnectivityInitialisation).

The ModelSpec::addSynapsePopulation() function returns a pointer to the newly created SynapseGroup object which can be further configured, namely with:
- SynapseGroup::setMaxConnections() and SynapseGroup::setMaxSourceConnections() to configure the maximum number of rows and columns respectively allowed in the synaptic matrix - this can improve performance and reduce memory usage when using SynapseMatrixConnectivity::SPARSE connectivity (see \ref subsect34).
\note
When using a sparse connectivity initialisation snippet, these values are set automatically.

- SynapseGroup::setMaxDendriticDelayTimesteps() sets the maximum dendritic delay (in terms of the simulation
     time step `DT`) allowed for synapses in this population. No values larger than this should be passed to the delay parameter of the `addToDenDelay` function in user code (see \ref sect34).
- SynapseGroup::setSpanType() sets how incoming spike processing is parallelised for this synapse group. The default SynapseGroup::SpanType::POSTSYNAPTIC is nearly always the best option, but SynapseGroup::SpanType::PRESYNAPTIC may perform better when there are large numbers of spikes every timestep or very few postsynaptic neurons.

\note
If the synapse matrix uses one of the "GLOBALG" types then the global
value of the synapse parameters are taken from the initial value provided
in `weightVarInitialisers` therefore these must be constant rather than sampled from a distribution etc.



-----
\link UserManual Previous\endlink | \link sectDefiningNetwork Top\endlink | \link sectNeuronModels Next\endlink
*/


//----------------------------------------------------------------------------
/*! 
\page sectNeuronModels Neuron models
There is a number of predefined models which can be used with the ModelSpec::addNeuronGroup function:
- NeuronModels::RulkovMap
- NeuronModels::Izhikevich
- NeuronModels::IzhikevichVariable
- NeuronModels::LIF
- NeuronModels::SpikeSource
- NeuronModels::PoissonNew
- NeuronModels::TraubMiles
- NeuronModels::TraubMilesFast
- NeuronModels::TraubMilesAlt
- NeuronModels::TraubMilesNStep

\section sect_own Defining your own neuron type 

In order to define a new neuron type for use in a GeNN application,
it is necessary to define a new class derived from NeuronModels::Base. 
For convenience the methods this class should implement can be implemented using macros:

- DECLARE_MODEL(TYPE, NUM_PARAMS, NUM_VARS): declared the boilerplate code required for the model e.g. the correct
  specialisations of NewModels::ValueBase used to wrap the neuron model parameters and values.
- SET_SIM_CODE(SIM_CODE): where SIM_CODE contains the code for executing 
  the integration of the model for one time stepWithin this code string, variables need to be
  referred to by \$(NAME), where NAME is the name of the variable as
  defined in the vector varNames. The code may refer to the predefined
  primitives `DT` for the
  time step size and `$(Isyn)` for the total incoming synaptic current. It can also refer to a unique ID (within the population) using $(id).
- SET_THRESHOLD_CONDITION_CODE(THRESHOLD_CONDITION_CODE) defines the condition for true spike detection.
- SET_PARAM_NAMES() defines the names of the model parameters. 
    If defined as `NAME` here, they can then be referenced as \$(NAME) in the code string. 
    The length of this list should match the NUM_PARAM specified in DECLARE_MODEL. 
    Parameters are assumed to be always of type double.
- SET_VARS() defines the names, type strings (e.g. "float", "double", etc) and (optionally) access mode
    of the neuron state variables. The type string "scalar" can be used for variables which should be implemented using the precision set globally for the model with ModelSpec::setPrecision.
    The variables defined here as `NAME` can then be used in the
    syntax \$(NAME) in the code string. If the access mode is set to ``VarAccess::READ_ONLY``, GeNN applies additional optimisations and models should not write to it.
- SET_NEEDS_AUTO_REFRACTORY() defines whether the neuron should include an automatic refractory period to prevent it emitting spikes in successive timesteps.

For example, using these macros, we can define a leaky integrator \f$\tau\frac{dV}{dt}= -V + I_{{\rm syn}}\f$ solved using Euler's method:

\code
class LeakyIntegrator : public NeuronModels::Base
{
public:
    DECLARE_MODEL(LeakyIntegrator, 1, 1);
    
    SET_SIM_CODE("$(V)+= (-$(V)+$(Isyn))*(DT/$(tau));");
    
    SET_THRESHOLD_CONDITION_CODE("$(V) >= 1.0");
    
    SET_PARAM_NAMES({"tau"});
    
    SET_VARS({{"V", "scalar", VarAccess::READ_WRITE}});
};
\endcode

Additionally "dependent parameters" can be defined. Dependent parameters are a mechanism 
for enhanced efficiency when running neuron models. If parameters with model-side meaning, such as time
constants or conductances always appear in a certain combination in the model, then it is more efficient 
to pre-compute this combination and define it as a dependent parameter.

For example, because the equation defining the previous leaky integrator example has an algebraic solution, it can be more accurately solved as follows - using a derived parameter to calculate \f$\exp\left(\frac{-t}{\tau}\right)\f$:

\code
class LeakyIntegrator2 : public NeuronModels::Base
{
public:
    DECLARE_MODEL(LeakyIntegrator2, 1, 1);
    
    SET_SIM_CODE("$(V) = $(Isyn) - $(ExpTC)*($(Isyn) - $(V));");
    
    SET_THRESHOLD_CONDITION_CODE("$(V) >= 1.0");
    
    SET_PARAM_NAMES({"tau"});
    
    SET_VARS({{"V", "scalar", VarAccess::READ_WRITE}});
    
    SET_DERIVED_PARAMS({
        {"ExpTC", [](const vector<double> &pars, double dt){ return std::exp(-dt / pars[0]); }}});
};
\endcode

GeNN provides several additional features that might be useful when defining more complex neuron models.
\subsection neuron_support_code Support code
Support code enables a code block to be defined that contains supporting code that will be utilized in multiple pieces of user code. Typically, these are functions that are needed in the sim code or threshold condition code. If possible, these should be defined as `__host__ __device__` functions so that both GPU and CPU versions of GeNN code have an appropriate support code function available. The support code is protected with a namespace so that it is exclusively available for the neuron population whose neurons define it. Support code is added to a model using the SET_SUPPORT_CODE() macro, for example:
\code
SET_SUPPORT_CODE("__device__ __host__ scalar mysin(float x){ return sin(x); }");
\endcode

\subsection neuron_extra_global_param Extra global parameters
Extra global parameters are parameters common to all neurons in the population. However, unlike the standard neuron parameters, they can be varied at runtime meaning they could, for example, be used to provide a global reward signal.
These parameters are defined by using the SET_EXTRA_GLOBAL_PARAMS() macro to specify a list of variable names and type strings (like the SET_VARS() macro). For example:
\code
SET_EXTRA_GLOBAL_PARAMS({{"R", "float"}});
\endcode

These variables are available to all neurons in the population. They can also be used in synaptic code snippets; in this case it need to be addressed with a `_pre` or `_post` postfix.

For example, if the model with the "R" parameter was used for the pre-synaptic neuron population, the weight update model of a synapse population could have simulation code like:
\code
SET_SIM_CODE("$(x)= $(x)+$(R_pre);");
\endcode
where we have assumed that the weight update model has a variable `x` and our synapse type will only be used in conjunction with pre-synaptic neuron populations that do have the extra global parameter `R`. If the pre-synaptic population does not have the required variable/parameter, GeNN will fail when compiling the kernels.

\subsection neuron_additional_input Additional input variables
Normally, neuron models receive the linear sum of the inputs coming from all of their synaptic inputs through the \$(inSyn) variable. 
However neuron models can define additional input variables - allowing input from different synaptic inputs to be combined non-linearly.
For example, if we wanted our leaky integrator to operate on the the product of two input currents, it could be defined as follows:
\code
SET_ADDITIONAL_INPUT_VARS({{"Isyn2", "scalar", 1.0}});
SET_SIM_CODE("const scalar input = $(Isyn) * $(Isyn2);\n"
             "$(V) = input - $(ExpTC)*(input - $(V));");
\endcode
Where the SET_ADDITIONAL_INPUT_VARS() macro defines the name, type and its initial value before postsynaptic inputs are applyed (see section \ref sect_postsyn for more details).

\subsection neuron_rng Random number generation
Many neuron models have probabilistic terms, for example a source of noise or a probabilistic spiking mechanism. In GeNN this can be implemented by using the following functions in blocks of model code:
- <code>\$(gennrand_uniform)</code> returns a number drawn uniformly from the interval \f$[0.0, 1.0]\f$
- <code>\$(gennrand_normal)</code> returns a number drawn from a normal distribution with a mean of 0 and a standard deviation of 1.
- <code>\$(gennrand_exponential)</code> returns a number drawn from an exponential distribution with \f$\lambda=1\f$.
- <code>\$(gennrand_log_normal, MEAN, STDDEV)</code> returns a number drawn from a log-normal distribution with the specified mean and standard deviation.
- <code>\$(gennrand_gamma, ALPHA)</code> returns a number drawn from a gamma distribution with the specified shape.

Once defined in this way, new neuron models classes, can be used in network descriptions by referring to their type e.g.
\code
networkModel.addNeuronPopulation<LeakyIntegrator>("Neurons", 1, 
                                                  LeakyIntegrator::ParamValues(20.0), // tau
                                                  LeakyIntegrator::VarValues(0.0)); // V
\endcode


-----
\link sectDefiningNetwork Previous\endlink | \link UserManual Top\endlink | \link sectSynapseModels Next\endlink
*/

//----------------------------------------------------------------------------
/*! 
\page sectSynapseModels Weight update models

Currently 4 predefined weight update models are available: 
- WeightUpdateModels::StaticPulse
- WeightUpdateModels::StaticPulseDendriticDelay
- WeightUpdateModels::StaticGraded
- WeightUpdateModels::PiecewiseSTDP

For more details about these built-in synapse models, see \cite Nowotny2010.

\section sect34 Defining a new weight update model

Like the neuron models discussed in \ref sect_own, new weight update models are created by defining a class.
Weight update models should all be derived from WeightUpdateModel::Base and, for convenience, the methods a new weight update model should implement can be implemented using macros:

- SET_DERIVED_PARAMS(), SET_PARAM_NAMES(), SET_VARS() and SET_EXTRA_GLOBAL_PARAMS() perform the same roles as they do in the neuron
  models discussed in \ref sect_own.
- DECLARE_WEIGHT_UPDATE_MODEL(TYPE, NUM_PARAMS, NUM_VARS, NUM_PRE_VARS, NUM_POST_VARS) is an extended version of ``DECLARE_MODEL()`` which declares the boilerplate code required for a weight update model with pre and postsynaptic as well as per-synapse state variables.
- SET_PRE_VARS() and SET_POST_VARS() define state variables associated with pre or postsynaptic neurons rather than synapses. These are typically used to efficiently implement <i>trace</i> variables for use in STDP learning rules \cite Morrison2008. Like other state variables, variables defined here as `NAME` can be accessed in weight update model code strings using the \$(NAME) syntax. 
- SET_SIM_CODE(SIM_CODE): defines the simulation code that is used when a true spike is detected. The update is performed only in timesteps after a neuron in the presynaptic population has fulfilled its threshold detection condition.
Typically, spikes lead to update of synaptic variables that then lead to the activation of input into the post-synaptic neuron. Most of the time these inputs add linearly at the post-synaptic neuron. This is assumed in GeNN and the term to be added to the activation of the post-synaptic neuron should be applied using the the \$(addToInSyn, weight) function. 
For example
\code
SET_SIM_CODE(
    "$(addToInSyn, $(inc));\n"
\endcode
where "inc" is the increment of the synaptic input to a post-synaptic neuron for each pre-synaptic spike. The simulation code also typically contains updates to the internal synapse variables that may have contributed to $(inc). For an example, see WeightUpdateModels::StaticPulse for a simple synapse update model and WeightUpdateModels::PiecewiseSTDP for a more complicated model that uses STDP. 
To apply input to the post-synaptic neuron with a dendritic (i.e. between the synapse and the postsynaptic neuron) delay you can instead use the \$(addToInSynDelay, weight, delay) function.
For example
\code
SET_SIM_CODE(
    "$(addToInSynDelay, $(inc), $(delay));");
\endcode
where, once again, `inc` is the magnitude of the input step to apply and `delay` is the length of the dendritic delay in timesteps. By implementing `delay` as a weight update model variable, heterogeneous synaptic delays can be implemented. For an example, see WeightUpdateModels::StaticPulseDendriticDelay for a simple synapse update model with heterogeneous dendritic delays. 
\note
When using dendritic delays, the <b>maximum</b> dendritic delay for a synapse populations must be specified using the `SynapseGroup::setMaxDendriticDelayTimesteps()` function.

- SET_EVENT_THRESHOLD_CONDITION_CODE(EVENT_THRESHOLD_CONDITION_CODE) defines a condition for a synaptic event. This typically involves the pre-synaptic variables, e.g. the membrane potential: 
\code
SET_EVENT_THRESHOLD_CONDITION_CODE("$(V_pre) > -0.02");
\endcode
Whenever this expression evaluates to true, the event code set using the SET_EVENT_CODE() macro is executed. For an example, see WeightUpdateModels::StaticGraded.

- SET_EVENT_CODE(EVENT_CODE) defines the code that is used when the event threshold condition is met (as set using the SET_EVENT_THRESHOLD_CONDITION_CODE() macro). 

- SET_LEARN_POST_CODE(LEARN_POST_CODE) defines the code which is used in the learnSynapsesPost kernel/function, which performs updates to synapses that are triggered by post-synaptic spikes. This is typically used in STDP-like models e.g. WeightUpdateModels::PiecewiseSTDP.

- SET_SYNAPSE_DYNAMICS_CODE(SYNAPSE_DYNAMICS_CODE) defines code that is run for each synapse, each timestep i.e. unlike the others it is not event driven. This can be used where synapses have internal variables and dynamics that are described in continuous time, e.g. by ODEs. However using this mechanism is typically computationally very costly because of the large number of synapses in a typical network. By using the \$(addtoinsyn), \$(updatelinsyn) and \$(addToDenDelay) mechanisms discussed in the context of SET_SIM_CODE(), the synapse dynamics can also be used to implement continuous synapses for rate-based models.
- SET_PRE_SPIKE_CODE() and SET_POST_SPIKE_CODE() define code that is called whenever there is a pre or postsynaptic spike. Typically these code strings are used to update any pre or postsynaptic state variables.
- SET_NEEDS_PRE_SPIKE_TIME(PRE_SPIKE_TIME_REQUIRED) and SET_NEEDS_POST_SPIKE_TIME(POST_SPIKE_TIME_REQUIRED) define whether the weight update needs to know the times of the spikes emitted from the pre and postsynaptic populations. For example an STDP rule would be likely to require:
\code
SET_NEEDS_PRE_SPIKE_TIME(true);
SET_NEEDS_POST_SPIKE_TIME(true);
\endcode

All code snippets, aside from those defined with ``SET_PRE_SPIKE_CODE()`` and ``SET_POST_SPIKE_CODE()``, can be used to manipulate any synapse variable and so learning rules can combine both time-drive and event-driven processes.


-----
\link sectNeuronModels Previous\endlink | \link UserManual Top\endlink | \link sect_postsyn Next\endlink
*/

//----------------------------------------------------------------------------
/*! 

\page sect_postsyn Postsynaptic integration methods

There are currently 3 built-in postsynaptic integration methods:
- PostsynapticModels::ExpCurr
- PostsynapticModels::ExpCond
- PostsynapticModels::DeltaCurr

\section sect_new_postsynaptic Defining a new postsynaptic model
The postsynaptic model defines how synaptic activation translates into an input current (or other input term for models that are not current based). It also can contain equations defining dynamics that are applied to the (summed) synaptic activation, e.g. an exponential decay over time.

In the same manner as to both the neuron and weight update models discussed in \ref sect_own and \ref sect34, postsynamic model definitions are encapsulated in a class derived from PostsynapticModels::Base. Again, the methods that a postsynaptic model should implement can be implemented using the following macros:
- DECLARE_MODEL(TYPE, NUM_PARAMS, NUM_VARS), SET_DERIVED_PARAMS(), SET_PARAM_NAMES(), SET_VARS() perform the same roles as they do in the neuron models discussed in \ref sect_own.
- SET_DECAY_CODE(DECAY_CODE) defines the code which provides the continuous time dynamics for the summed presynaptic inputs to the postsynaptic neuron. This usually consists of some kind of decay function.
- SET_APPLY_INPUT_CODE(APPLY_INPUT_CODE) defines the code specifying the conversion from synaptic inputs to a postsynaptic neuron input current. e.g. for a conductance model:
\code
SET_APPLY_INPUT_CODE("$(Isyn) += $(inSyn) * ($(E) - $(V))");
\endcode
where \$(E) is a postsynaptic model parameter specifying reversal potential and \$(V) is the variable containing the postsynaptic neuron's membrane potential.
As discussed in \ref predefinedVars, \$(Isyn) is the built in variable used to sum neuron input.
However additional input variables can be added to a neuron model using the SET_ADDITIONAL_INPUT_VARS() macro (see \ref sect_own for more details).


-----
\link sectNeuronModels Previous\endlink | \link UserManual Top\endlink | \link sectCurrentSourceModels Next\endlink
*/

//----------------------------------------------------------------------------
/*! 
\page sectCurrentSourceModels Current source models
There is a number of predefined models which can be used with the ModelSpec::addCurrentSource function:
- CurrentSourceModels::DC
- CurrentSourceModels::GaussianNoise

\section sect_own_current_source Defining your own current source model 

In order to define a new current source type for use in a GeNN application,
it is necessary to define a new class derived from CurrentSourceModels::Base. 
For convenience the methods this class should implement can be implemented using macros:

- DECLARE_MODEL(TYPE, NUM_PARAMS, NUM_VARS), SET_DERIVED_PARAMS(), SET_PARAM_NAMES(), SET_VARS() perform the same roles as they do in the neuron models discussed in \ref sect_own.
- SET_INJECTION_CODE(INJECTION_CODE): where INJECTION_CODE contains the code for injecting current into the neuron every simulation timestep.
  The \$(injectCurrent, $(amp)) function is used to inject current.

For example, using these macros, we can define a uniformly distributed noisy current source:

\code
class UniformNoise : public CurrentSourceModels::Base
{
public:
    DECLARE_MODEL(UniformNoise, 1, 0);
    
    SET_SIM_CODE("$(injectCurrent, $(gennrand_uniform) * $(magnitude));");
    
    SET_PARAM_NAMES({"magnitude"});
};
\endcode


-----
\link sect_postsyn Previous\endlink | \link UserManual Top\endlink | \link subsect34 Next\endlink
*/

//----------------------------------------------------------------------------
/*! 
\page subsect34 Synaptic matrix types

Synaptic matrix types are made up of two components: SynapseMatrixConnectivity and SynapseMatrixWeight.
SynapseMatrixConnectivity defines what data structure is used to store the synaptic matrix:
- SynapseMatrixConnectivity::DENSE stores synaptic matrices as a dense matrix. Large dense matrices require a large amount of memory and if they contain a lot of zeros it may be inefficient.
- SynapseMatrixConnectivity::SPARSE stores synaptic matrices in a(padded) 'ragged array' format. In general, this is less efficient to traverse using a GPU than the dense matrix format but does result in significant memory savings for large matrices.
Ragged matrix connectivity is stored using several variables whose names, like state variables, have the name of the synapse population appended to them:
        -# `const unsigned int maxRowLength`: a constant set via the ``SynapseGroup::setMaxConnections`` method which specifies the maximum number of connections in any given row (this is the width the structure is padded to). 
        -# `unsigned int *rowLength` (sized to number of presynaptic neurons): actual length of the row of connections associated with each presynaptic neuron
        -# `unsigned int *ind` (sized to ``maxRowLength * number of presynaptic neurons``): Indices of corresponding postsynaptic neurons concatenated for each presynaptic neuron.
For example, consider a network of two presynaptic neurons connected to three postsynaptic neurons: 0th presynaptic neuron connected to 1st and 2nd postsynaptic neurons, the 1st presynaptic neuron connected only to the 0th neuron. The struct RaggedProjection should have these members, with indexing from 0 (where X represents a padding value):
\code
maxRowLength = 2
ind = [1 2 0 X]
rowLength = [2 1]  
\endcode
Weight update model variables associated with the sparsely connected synaptic population will be kept in an array using the same indexing as ind. For example, a variable caled \c g will be kept in an array such as:
\c g=[g_Pre0-Post1 g_pre0-post2 g_pre1-post0 X]
- SynapseMatrixConnectivity::BITMASK is an alternative sparse matrix implementation where which synapses within the matrix are present is specified as a binary array (see \ref ex_mbody). This structure is somewhat less efficient than the ``SynapseMatrixConnectivity::SPARSE`` and ``SynapseMatrixConnectivity::RAGGED`` formats and doesn't allow individual weights per synapse. However it does require the smallest amount of GPU memory for large networks.
 
Furthermore the SynapseMatrixWeight defines how 
- SynapseMatrixWeight::INDIVIDUAL allows each individual synapse to have unique weight update model variables. 
Their values must be initialised at runtime and, if running on the GPU, copied across from the user side code, using the \c pushXXXXXStateToDevice function, where XXXX is the name of the synapse population.
- SynapseMatrixWeight::INDIVIDUAL_PSM allows each postsynapic neuron to have unique post synaptic model variables.
Their values must be initialised at runtime and, if running on the GPU, copied across from the user side code, using the \c pushXXXXXStateToDevice function, where XXXX is the name of the synapse population.
- SynapseMatrixWeight::GLOBAL saves memory by only maintaining one copy of the weight update model variables.
This is automatically initialized to the initial value passed to ModelSpec::addSynapsePopulation.

Only certain combinations of SynapseMatrixConnectivity and SynapseMatrixWeight are sensible therefore, to reduce confusion, the SynapseMatrixType enumeration defines the following options which can be passed to ModelSpec::addSynapsePopulation:
- SynapseMatrixType::SPARSE_GLOBALG
- SynapseMatrixType::SPARSE_GLOBALG_INDIVIDUAL_PSM
- SynapseMatrixType::SPARSE_INDIVIDUALG
- SynapseMatrixType::DENSE_GLOBALG
- SynapseMatrixType::DENSE_GLOBALG_INDIVIDUAL_PSM
- SynapseMatrixType::DENSE_INDIVIDUALG
- SynapseMatrixType::BITMASK_GLOBALG
- SynapseMatrixType::BITMASK_GLOBALG_INDIVIDUAL_PSM


-----
\link sectCurrentSourceModels Previous\endlink | \link UserManual Top\endlink | \link sectVariableInitialisation Next\endlink
*/
//----------------------------------------------------------------------------
/*! 
\page sectVariableInitialisation Variable initialisation

Neuron, weight update and postsynaptic models all have state variables which GeNN can automatically initialise. 

Previously we have shown variables being initialised to constant values such as:
\code
NeuronModels::TraubMiles::VarValues ini(
    0.0529324,     // 1 - prob. for Na channel activation m
    ...
);
\endcode
state variables can also be left _uninitialised_ leaving it up to the user code to initialise them between the calls to ``initialize()`` and ``initializeSparse()``:
\code
NeuronModels::TraubMiles::VarValues ini(
    uninitialisedVar(),     // 1 - prob. for Na channel activation m
    ...
);
\endcode
or initialised using one of a number of predefined _variable initialisation snippets_:
- InitVarSnippet::Uniform
- InitVarSnippet::Normal
- InitVarSnippet::Exponential
- InitVarSnippet::Gamma

For example, to initialise a parameter using values drawn from the normal distribution:
\code
InitVarSnippet::Normal::ParamValues params(
    0.05,   // 0 - mean
    0.01);  // 1 - standard deviation
    
NeuronModels::TraubMiles::VarValues ini(
    initVar<InitVarSnippet::Normal>(params),     // 1 - prob. for Na channel activation m
    ...
);
\endcode
\section sect_new_var_init Defining a new variable initialisation snippet
Similarly to neuron, weight update and postsynaptic models, new variable initialisation snippets can be created by simply defining a class in the model description.
For example, when initialising excitatory (positive) synaptic weights with a normal distribution they should be clipped at 0 so the long tail of the normal distribution doesn't result in negative weights.
This could be implemented using the following variable initialisation snippet which redraws until samples are within the desired bounds:
\code
class NormalPositive : public InitVarSnippet::Base
{
public:
    DECLARE_SNIPPET(NormalPositive, 2);

    SET_CODE(
        "scalar normal;"
        "do\n"
        "{\n"
        "   normal = $(mean) + ($(gennrand_normal) * $(sd));\n"
        "} while (normal < 0.0);\n"
        "$(value) = normal;\n");

    SET_PARAM_NAMES({"mean", "sd"});
};
IMPLEMENT_SNIPPET(NormalPositive);
\endcode
Within the snippet of code specified using the `SET_CODE()` macro, when initialisising neuron and postaynaptic model state variables , the \$(id) variable can be used to access the id of the neuron being initialised.
Similarly, when initialising weight update model state variables, the \$(id_pre) and \$(id_post) variables can used to access the ids of the pre and postsynaptic neurons connected by the synapse being initialised.

\section sect_var_init_modes Variable locations
Once you have defined <b>how</b> your variables are going to be initialised you need to configure <b>where</b> they will be allocated. 
By default memory is allocated for variables on both the GPU and the host.
However, the following alternative 'variable locations' are available:
- VarLocation::DEVICE - Variables are only allocated on the GPU, saving memory but meaning that they can't easily be copied to the host - best for internal state variables.
- VarLocation::HOST_DEVICE - Variables are allocated on both the GPU and the host  - the default.
- VarLocation::HOST_DEVICE_ZERO_COPY - Variables are allocated as 'zero-copy' memory accessible to the host and GPU - useful on devices such as Jetson TX1 where physical memory is shared between the GPU and CPU.

\note
'Zero copy' memory is only supported on newer embedded systems such as the Jetson TX1 where there is no physical seperation between GPU and host memory and thus the same block of memory can be shared between them. 

These modes can be set as a model default using ``ModelSpec::setDefaultVarLocation`` or on a per-variable basis using one of the following functions:
- NeuronGroup::setSpikeLocation
- NeuronGroup::setSpikeEventLocation
- NeuronGroup::setSpikeTimeLocation
- NeuronGroup::setVarLocation
- SynapseGroup::setWUVarLocation
- SynapseGroup::setWUPreVarLocation
- SynapseGroup::setWUPostVarLocation
- SynapseGroup::setPSVarLocation
- SynapseGroup::setInSynVarLocation


-----
\link subsect34 Previous\endlink | \link UserManual Top\endlink | \link sectSparseConnectivityInitialisation Next\endlink
*/
//----------------------------------------------------------------------------
/*! 
\page sectSparseConnectivityInitialisation Sparse connectivity initialisation

Synaptic connectivity implemented using SynapseMatrixConnectivity::SPARSE and SynapseMatrixConnectivity::BITMASK can be automatically initialised. 

This can be done using one of a number of predefined _sparse connectivity initialisation snippets_:
- InitSparseConnectivitySnippet::OneToOne
- InitSparseConnectivitySnippet::FixedProbability
- InitSparseConnectivitySnippet::FixedProbabilityNoAutapse

For example, to initialise synaptic connectivity with a 10% connection probability (allowing connections between neurons with the same id):
\code
InitSparseConnectivitySnippet::FixedProbability::ParamValues fixedProb(0.1);
    
model.addSynapsePopulation<...>(
        ...
        initConnectivity<InitSparseConnectivitySnippet::FixedProbability>(fixedProb));
\endcode

\section sect_new_sparse_connect Defining a new sparse connectivity snippet
Similarly to variable initialisation snippets, sparse connectivity initialisation snippets can be created by simply defining a class in the model description.

For example, the following sparse connectivity initialisation snippet could be used to initialise a 'ring' of connectivity where each neuron is connected to a number of subsequent neurons specified using the ``numNeighbours`` parameter:
\code
class Ring : public InitSparseConnectivitySnippet::Base
{
public:
    DECLARE_SNIPPET(Ring, 1);

    SET_ROW_BUILD_STATE_VARS({{"offset", {"unsigned int", 1}}}});
    SET_ROW_BUILD_CODE(
        "const unsigned int target = ($(id_pre) + offset) % $(num_post);\n"
        "$(addSynapse, target);\n"
        "offset++;\n"
        "if(offset > (unsigned int)$(numNeighbours)) {\n"
        "   $(endRow);\n"
        "}\n");

    SET_PARAM_NAMES({"numNeighbours"});
    SET_CALC_MAX_ROW_LENGTH_FUNC(
        [](unsigned int numPre, unsigned int numPost, const std::vector<double> &pars)
        {
            return (unsigned int)pars[0];
        });
    SET_CALC_MAX_COL_LENGTH_FUNC(
        [](unsigned int numPre, unsigned int numPost, const std::vector<double> &pars)
        {
            return (unsigned int)pars[0];
        });
};
IMPLEMENT_SNIPPET(Ring);
\endcode
Each <i>row</i> of sparse connectivity is initialised independantly by running the snippet of code specified using the `SET_ROW_BUILD_CODE()` macro within a loop.
The \$(num_post) variable can be used to access the number of neurons in the postsynaptic population and the \$(id_pre) variable can be used to access the index of the presynaptic neuron associated with the row being generated.
The `SET_ROW_BUILD_STATE_VARS()` macro can be used to initialise state variables outside of the loop - in this case ``offset`` which is used to count the number of synapses created in each row.
Synapses are added to the row using the \$(addSynapse, target) function and iteration is stopped using the \$(endRow) function.
To avoid having to manually call SynapseGroup::setMaxConnections and SynapseGroup::setMaxSourceConnections, sparse connectivity snippets can also provide code to calculate the maximum row and column lengths this connectivity will result in using the SET_CALC_MAX_ROW_LENGTH_FUNC() and SET_CALC_MAX_COL_LENGTH_FUNC() macros.
Alternatively, if the maximum row or column length is constant, the `SET_MAX_ROW_LENGTH()` and `SET_MAX_COL_LENGTH()` shorthand macros can be used.

\section sect_sparse_connect_init_modes Sparse connectivity locations
Once you have defined <b>how</b> sparse connectivity is going to be initialised, similarly to variables, you can control <b>where</b> it is allocated. 
This is controlled using the same ``VarLocations`` options described in section \ref sect_var_init_modes and can either be set using the model default specifiued with ``ModelSpec::setDefaultSparseConnectivityLocation`` or on a per-synapse group basis using ``SynapseGroup::setSparseConnectivityLocation``.


-----
\link sectVariableInitialisation Previous\endlink | \link UserManual Top\endlink | \link Tutorial1 Next\endlink
*/