turbo_seti Maintenance & Regression Testing
===========================================


### Introduction

The purpose of the regression testing suite is to exercise and validate results from turbo_seti functional modules.  This is important in order to minimize potential inadvertent breakage when new development has occured. It is always best to catch bugs as soon as possible after they are introduced.
<br>
The primary method of launching regression testing is through the use of the `pytest` executable.  This is invoked in the following ways:
* Manually by a developer, on the command line in a terminal window.  This would follow downloading turbo_seti and setting up the development/testing environment (discussed later). 
* Automatically as part of a Github Pull Request (PR) after finalizing a fork of turbo_seti.
* Automatically as part of a Github Merge after a PR is approved.
<br>

### Development Process Overview

* The development of an amendment to `turbo_seti` begins with taking a fork from a github site, normally from `https://github.com/UCBerkeleySETI/turbo_seti`.
* Also, from the same site, `turbo_seti` is downloaded to a local computer.  The download operations can be performed in a few different ways but the simplest might be to download the zip file by clicking on the `Code` button and selecting `Download ZIP`.  Once the the zip file is in a local directory, unzip it and move the turbo_seti directory tree to wherever is appropriate for testing.  The zip file can now be discarded.
* Change directory into the `test` directory and execute `python3 download_test_data.py` which will perform all required regression testing initialization.
* When the previous step has completed, change directory up one level to the top of the `turbo_seti` directory tree.
* Execute: ```python3 -m pip install -U git+https://github.com/UCBerkeleySETI/blimpy```
* Execute: ```python3 setup.py install```
* Execute: ```python3 -m pip install -r requirements_test.txt```
* Regression testing can now begin.
* Running the full suite of regression tests is invoked by executing `pytest` with no parameters specified.  It is possible to run a single regression test file by specifying it as an argument to `pytest`.  For example, if one wishes to only run the find event tests, the following is the command line to use: `pytest test/test_find_event.py`.
* It is **highly encouraged** for developers to perform regression testing frequently in order to avoid surprises later on.
* Once, development activity on the local machine is complete and the last regression test has run verifying the absence of negative side effects, then the new and/or modified turbo_seti files can be uploaded to the developer's fork github site.
* Be sure to bump the version in `setup.py` and add an entry to `VERSION-HISTORY.md`.
* At the fork github site, the developer can request a pull clicking on the `Pull request` button.  This automatically starts the PR process mentioned in the introduction section.

### Testing Artifacts

|    File    | Description |
| :-- | :-- |
| `download_test_data.py` | Initialization of regression testing by obtaining and producing files used in test suite execution. |
| `fb_cases_def.py` | Data definitions for `test_fb_cases.py`. |
| `fb_cases_util.py` | Utility functions for `test_fb_cases.py` and `test_pipelines_{1,2,3}`. |
| `fb_dat_reference.txt` | Reference data generated by `fb_genref.py`, hopefully a one-time event. |
| `fb_genref.py` | Hopefully, a one-time generator of `fb_dat_reference.txt` which is used by `test_fb_cases.py` to validate test results.  The reference data values came from an execution of `turbo_seti` version 1.3.0. |
| `pipelines_util.py` | Utility functions for `test_pipelines_{1,2,3}`. |
| `README.md` | This file. |
| `run_benchmark.sh` | A bash script used for measuring performance in the Seti BL data centre. |
| `run_tests.sh` | A bash script used in Github Actions. |
| `test_drift_rates.py` | Test source for generating multiple min_drift and max_drift tests, employing the `setigen` tool. |
| `test_fb_cases.py` | Test source for generating multiple Filterbank tests, employing the `setigen` tool. |
| `test_find_event.py` | Test source for generating multiple tests of the `find_event.py` module. |
| `test_pipelines_1.py` | Test source for exercising/validation of `find_event_pipeline.py` and `plot_event_pipeline.py`. |
| `test_pipelines_2.py` | Same as `test_pipelines_1.py` except it tests observation file cadences that start with "OFF" target instead of starting with "ON" target. |
| `test_pipelines_3.py` | Test source for exercising/validation of `find_event_pipeline.py` when the collection of input files are a complex cadence. |
| `test_pipelines_4.py` | Test source for exercising/validation of `run_pipeline.py`. |
| `test_plot_dat.py` | Test source for exercising/validation of `plot_dat.py`. |
| `test_turbo_seti.py` | Test source for exercising/validation of a variety of turbo_seti functions. |