PyRate - a Python tool for InSAR Rate and Time-series Estimation
================================================================

.. image:: https://travis-ci.org/GeoscienceAustralia/PyRate.svg?branch=master
   :target: https://travis-ci.org/GeoscienceAustralia/PyRate
.. image:: https://coveralls.io/repos/github/GeoscienceAustralia/PyRate/badge.svg?branch=master
   :target: https://coveralls.io/github/GeoscienceAustralia/PyRate?branch=master

PyRate is a Python tool for estimating the average rate (velocity) and incremental time-series of surface movement for every pixel in a stack of images generated by interferometric processing of Synthetic Aperture Radar (InSAR) data. It has been designed to use GAMMA or ROI_PAC format geocoded unwrapped interferograms.

PyRate is a partial Python translation of `Pirate <http://homepages.see.leeds.ac.uk/~earhw/software/pirate/>`_, a Matlab tool developed by the University of Leeds and the Guangdong University of Technology.

Home Page: https://github.com/GeoscienceAustralia/PyRate

Documentation: http://geoscienceaustralia.github.io/PyRate

Issue tracking: https://github.com/GeoscienceAustralia/PyRate/issues

============
Installation
============

Before you start, you will need to have a number of packages installed on your Linux system. These can either be installed directly onto the system, or you can use a virtual environment.

**************
Direct Install
**************

Make sure your system has the following packages installed. If not, run the command:

.. code-block:: console

    sudo apt-get install gdal-bin libgdal-dev libpng12-dev libblas-dev liblapack-dev libatlas-dev libatlas-base-dev gfortran libproj-dev openmpi-bin libopenmpi-dev netcdf-bin libnetcdf11 libnetcdf-dev

*******************
Virtual Environment
*******************

You can use one of the two virtual environment options below:

Note: if you are using a HPC environment, refer to `these instructions instead <https://github.com/GeoscienceAustralia/PyRate/tree/master/hpc/README.rst>`_.


**Virtualenv**

1. Install `virtualenv <https://gist.github.com/basaks/b33ea9106c7d1d72ac3a79fdcea430eb>`_.
2. You may need to install a slightly older python-daemon to install PyRate:

.. code-block:: console

    pip install python-daemon==2.1.1

3. Install PyRate by using one of the following options.

- Run ``setup.py``:

.. code-block:: console

    python setup.py install

- Install the latest version of PyRate with ``pip`` from github:

.. code-block:: console

    pip install git+https://github.com/GeoscienceAustralia/PyRate

- PyRate is also on ``pypi``, the Python package manager. To install, run:

.. code-block:: console

    pip install Py-Rate

The Python requirements should automatically be built and installed.

**Anaconda**

For using PyRate in an Anaconda environment `use this
guide <https://github.com/GeoscienceAustralia/PyRate/blob/master/conda.md>`_.

=====
Tests
=====

A suite of tests have been developed for use in testing PyRate functionality
and for further code development. The tests use
`pytest <http://doc.pytest.org/en/latest/>`_ and can be found in the *tests/*
directory. A small test dataset is included in the *tests/test_data/*
directory.

To run the tests, use the following command inside the top level *PyRate/*
directory:

.. code-block:: console

    pip install pytest
    cd PyRate
    export PYRATEPATH=/path/to/PyRate
    pytest tests/

==================
Configuration File
==================

Example configuration files for running PyRate with GAMMA or ROI\_PAC format
interferograms are contained in the *configs/* directory.

===============
PyRate Workflow
===============

After following the steps under Installation, an executable program
``pyrate`` is created.

Use ``help`` for the different command line options:

.. code-block:: console

    >> pyrate --help
    Usage: pyrate [OPTIONS] COMMAND [ARGS]...

    Options:
      -v, --verbosity [DEBUG|INFO|WARNING|ERROR]
                                      Level of logging
      --help                          Show this message and exit.

    Commands:
      linrate
      postprocess
      prepifg

The ``pyrate`` program has three command line options corresponding to 
different parts of the PyRate workflow:

1. ``prepifg``
2. ``linrate``
3. ``postprocess``

Below we discuss these options.

******************************************
1. prepifg: Preparing input interferograms
******************************************

The first step of PyRate is to convert the GAMMA or ROI\_PAC format
unwrapped interferograms into geotiff format, followed by applying
multi-looking and cropping operations. These procedures are all
performed by the ``pyrate prepifg`` command:

.. code-block:: console

    >> pyrate prepifg --help
    Usage: pyrate prepifg [OPTIONS] CONFIG_FILE

    Options:
      --help  Show this message and exit.

The ``prepifg`` command is used as follows:

.. code-block:: console

    pyrate prepifg /path/to/config_file

The two major steps during the ``prepifg`` operation are described
below.

**Data formatting: convert to geotiff**

The ``prepifg`` command will determine the input format from the value
specified at the *processor:* keyword in the config file (0: ROI\_PAC;
1: GAMMA)

Each GAMMA geocoded unwrapped interferogram requires three header files to
extract metadata required for data formatting: a geocoded DEM header file
(*\*.dem.par*), and the master and slave epoch SLC parameter files
(*\*.slc.par*). 

The path and name of the DEM header file are specified in the config file
under the *demHeaderFile:* keyword. 

The SLC parameter files should be in the directory specified in the config
file under the *slcFileDir:* keyword. SLC parameter files for a particular
interferogram are found automatically by date-string pattern matching.

Each ROI_PAC geocoded unwrapped interferogram requires its own header/resource
file (*\*.unw.rsc*). These header files need to be stored in the same directory
as the interferograms.

In addition, the geocoded DEM header file (*\*.dem.rsc*) is required and its
path and name are specified in the config file under the *demHeaderFile:*
keyword. The geographic projection in the parameter *DATUM:* is extracted
from the DEM header file.

**Image transformations: multi-looking and cropping**

The ``prepifg`` command will also perform multi-looking (image
sub-sampling) and cropping of the input interferograms.

Two example configuration files are provided in the *configs/* directory, one
each for ROI_PAC and GAMMA prepifg configuration. Either configuration file
can be used with ``prepifg``.

******************************************************************
2. linrate: Main workflow and linear rate and time series analysis
******************************************************************

.. code-block:: python

    >> pyrate linrate --help
    Usage: pyrate linrate [OPTIONS] CONFIG_FILE

    Options:
      -r, --rows INTEGER  divide ifgs into this many rows
      -c, --cols INTEGER  divide ifgs into this many columns
      --help              Show this message and exit

This is the core of the PyRate processing workflow, handled by the ``linrate``
command:

.. code-block:: python

    pyrate linrate path/to/config_file -c 3 -r 4

This command will perform the time series and linear rate analysis and has
the option to break the interferograms into a number of tiles in ``r`` rows and
``c`` columns. For example, the above command will break the interferograms
into 12 tiles and will produce 12 linear rate and time series products 
corresponding to each tile.

The optional rows and columns arguments can be used to create smaller ``tiles``
of the full size interferograms. This enables large interferograms to be more
easily be accommodated in memory. The number of tiles chosen should be as small
as possible that fits in the system memory.

Optionally, an orbital error correction and a spatio-temporal filter operation
to estimate and remove atmospheric phase screen signals is applied to the
interferograms prior to time series and linear rate analysis.
The corrected interferograms are updated on disk and the corrections are not
re-applied upon subsequent runs. This functionality is controlled by the
*orbfit:* and *apsest:* options in the configuration file.

Non-optional pre-processing steps include:
- Minimum Spanning Tree matrix calculation
- Identification of a suitable reference pixel
- Removal of reference phase from interferograms
- Calculation of interferogram covariance
- Assembly of the variance-covariance matrix

***********************************************
3. postprocess: Putting the tiles back together
***********************************************

The last step of the PyRate workflow is to re-assemble the tiles and save 
geotiff files of the final time series and linear rate products.

.. code-block:: python

    >> pyrate postprocess --help
    Usage: pyrate postprocess [OPTIONS] CONFIG_FILE

    Options:
      -r, --rows INTEGER  divide ifgs into this many rows
      -c, --cols INTEGER  divide ifgs into this many columns
      --help              Show this message and exit.

Make sure to use the same number of rows and columns that was used in the 
previous ``linrate`` step:

.. code-block:: python

    pyrate postprocess path/to/config_file -c 3 -r 4

===========
MPI Support
===========

PyRate has been designed for use on High Performance Computers and
instructions to use a HPC cluster can be found in the `hpc directory <hpc>`_.

==============================
Python Multi-threading Support
==============================

In addition to the MPI support for HPC, PyRate can use standard
multi-threading simply by turning ``parallel:  1`` in the configuration file
to take advantage of multiple cores on a single PC.

===============
Bugs & Feedback
===============

For bugs, questions and discussions, please use `Github
Issues <https://github.com/GeoscienceAustralia/PyRate/issues>`_.