Redundancy
==========

If you have large files (>1Gb) and access to a cluster, take a look at :doc:`mpi`.

For the impatient
-----------------

Even if you're in a hurry, taking a look at :doc:`preprocess` is very important. If you already did, you can simply run::

    nonpareil -s reads.fa -T kmer -f fastq -b output (fastq is recommended for kmer algorithm)
    nonpareil -s reads.fa -T kmer -f fasta -b output
    nonpareil -s reads.fa -T alignment -f fasta -b output (fasta is recommended for alignment algorithm)
    nonpareil -s reads.fa -T alignment -f fastq -b output
    

Where ``reads.fa`` is the file containing the trimmed single reads, and ``output`` is the prefix
of the output files to be created.

Mandatory options
-----------------
   -s <str>   Path to the (input) file containing the sequences.  This is lowercase S.
   -T <str>   nonpareil algorithm. can be 'kmer' or 'alignment'.
   -f <str>   The format of the sequence. Can be 'fasta' or 'fastq'.

Common options
--------------
   -b <str>   Path to the prefix for all the output files.  Replaces the options: -a, -C, -l, and -o; generating files
              with the suffixes .npa, npc, .npl, and .npo, respectively, unless explicitly set.
   -d <num>   Subsample iteratively applying this factor to the number of reads, resulting in logarithmic subsampling.
              Use -d 0 to fall back to linear sampling, controlled by -m, -M, & -i (this was the default before v2.4).
	      By default: 0.7.
   -n <int>   Number of sub-samples to generate per point.  If it is not a multiple of the number of threads (see -t),
              it is rounded to the next (upper) multiple.  By default: 1024.
   -L <num>   Minimum overlapping percentage of the aligned region on the largest sequence. The similarity (see -S) is
              evaluated for the aligned region only.  By default: 50.
   -X <int>   Maximum number of reads to use as query.  This is capital X.  By default, 1,000 reads.
   -q <str>   Path to the (input) file containing a second dataset to be used as query, for dataset comparisons.  This
	      option is currently experimental.
   -R <int>   Maximum RAM usage in Mib.  Ideally this value should be larger than the sequences to analyze (discarding
              non-sequence elements like headers or quality).  This is particularly important when running in multiple
              cores (see -t).  This value is approximated.  By default 1024.
              Maximum value in this version: 4194303
   -t <int>   Number of threads.  Highest efficiency when the number of sub-samples (see -n) is multiple of the number
              of threads.  By default: 2.
   -v <int>   Verbosity level, for debugging purposes.  By default 7.  This is lowercase V.
   -V         Show version information and exit.  This is uppercase V.
   -h         Display this message and exit.

Additional options
------------------
**Input/Output**
   -a <str>   Path to the (output) file where all data must be saved.  This report is not created by default.  See the
              OUTPUT section.
   -C <str>   Path to the (output) file where the mating vector is to be saved.  This is a capital C.
   -F         Report the sampled portions as a fraction of the library instead of the number of reads.  See -a, -o and
              the OUTPUT section.
   -l <str>   Path to the (output) file where the log of the run must be saved. By default the log is sent only to the
              STDERR.  If set, the log is sent to both the STDERR and the log file.
   -o <str>   Path to the (output) file where summary is to be saved.   By default the summary is sent to stdout (same
              behavior as using a dash '-').  If an empty string '' is provided, does not produce the summary. See the
              OUTPUT section.
   
**Sampling**
   -m <num>   Minimum value of sampling portion.  By default: 0.
   -M <num>   Maximum value of sampling portion.  By default: 1.
   -i <num>   Interval between sampling portions. By default: 0.01.

**Mating**
   -c         Do not use reverse-complement.  This is useful for single stranded sequences data (like RNA).  This is a
              lowercase C.
   -N         Treat Ns as mismatches.  By default, Ns (unknown nucleotides) match any nucleotide (even another N).
   -S <num>   Similarity threshold to group two reads together.   Reducing this option will increase sensitivity while
              increasing running time.  This is uppercase S.
   -x <num>   Probability of taking a sequence into account as query for the construction of the curve.  Higher values
              reduce accuracy but increase speed.  This is lower case x.  If set, overides -X.

**Misc**
   -A         Autoadjust parameters and re-run.  Evaluates the results looking for common problems, adjusts parameters
              and re-run the analyses.  THIS IS EXPERIMENTAL CODE.
   -r <int>   Random generator seed.  By default current time.

Input
-----
Sequences must be in FastA or FastQ format. See :doc:`preprocess`.

Output
------
Redundancy summary: ``.npo`` file
   Tab-delimited file with six columns. The first column indicates the sequencing effort (in number of reads), and the
   remaining columns indicate the summary of the distribution of redundancy (from the replicates, 1,024 by default) at
   the given sequencing effort. These five columns are: average redundancy, standard deviation, quartile 1, median
   (quartile 2), and quartile 3.

Redundancy values: ``.npa`` file
   Tab-delimited file with three columns. Similar to the .npo files, it contains information about the redundancy at
   each sequencing effort, but it provides ALL the results from the replicates, not only the summary at each point. The
   first column indicates the sequencing effort (as a fraction of the dataset), the second column indicates the ID of
   the replicate (a number used only to introduce some controlled noise in plots), and the third column indicates the
   estimated redundancy value.

Mates distribution: ``.npc`` file
   Raw list with the number of reads in the dataset matching a query read. A set of query reads is randomly drawn by
   Nonpareil (1,000 by default), and compared against all reads in the dataset. Each line on this file corresponds to a
   query read (the order is not important). We have seen certain correspondance between these numbers and the distribution
   of abundances in the community (compared, for example, as rank-abundance plots), but this file is provided only for
   quality-control purposes and comparisons with other tools.

Log: ``.npl`` file
   A verbose log of internal Nonpareil processing. The number to the left (inside squared brackets) indicate the CPU time
   (in minutes). This file also provide quality assessment of the Nonpareil run (automated consistency evaluation). Ideally,
   the last line should read "Everything seems correct". Otherwise, it suggests alternative parameters that may improve the
   estimation.