Package: ff
Version: 2.1-2
Date: 2008-08-03
Title: memory-efficient storage of large data on disk and fast access
        functions
Author: Daniel Adler <dadler@uni-goettingen.de>, Christian Gl\xE4ser
        <christian_glaeser@gmx.de>, Oleg Nenadic
        <onenadi@uni-goettingen.de>, Jens Oehlschl\xE4gel
        <Jens.Oehlschlaegel@truecluster.com>, Walter Zucchini
        <wzucchi@uni-goettingen.de>
Maintainer: Jens Oehlschl\xE4gel <Jens.Oehlschlaegel@truecluster.com>
Depends: R (>= 2.9.2), utils, tools, bit (>= 1.1-1)
Suggests: biglm
Description: The ff package provides data structures that are stored on
        disk but behave (almost) as if they were in RAM by
        transparently mapping only a section (pagesize) in main memory
        - the effective virtual memory consumption per ff object. ff
        supports R's standard atomic data types 'double', 'logical',
        'raw' and 'integer' and non-standard atomic types boolean (1
        bit), quad (2 bit unsigned), nibble (4 bit unsigned), byte (1
        byte signed with NAs), ubyte (1 byte unsigned), short (2 byte
        signed with NAs), ushort (2 byte unsigned), single (4 byte
        float with NAs). For example 'quad' allows efficient storage of
        genomic data as an 'A','T','G','C' factor. The unsigned types
        support 'circular' arithmetic. There is also support for
        close-to-atomic types 'factor', 'ordered', 'POSIXct', 'Date'
        and custom close-to-atomic types. ff not only has native
        C-support for vectors, matrices and arrays with flexible
        dimorder (major column-order, major row-order and
        generalizations for arrays). There is also a ffdf class not
        unlike data.frames and import/export filters for csv files. ff
        objects store raw data in binary flat files in native encoding,
        and complement this with metadata stored in R as physical and
        virtual attributes. ff objects have well-defined hybrid copying
        semantics, which gives rise to certain performance improvements
        through virtualization. ff objects can be stored and reopened
        across R sessions. ff files can be shared by multiple ff R
        objects (using different data en/de-coding schemes) in the same
        process or from multiple R processes to exploit parallelism. A
        wide choice of finalizer options allows to work with
        'permanent' files as well as creating/removing 'temporary' ff
        files completely transparent to the user. On certain
        OS/Filesystem combinations, creating the ff files works without
        notable delay thanks to using sparse file allocation. Several
        access optimization techniques such as Hybrid Index
        Preprocessing and Virtualization are implemented to achieve
        good performance even with large datasets, for example virtual
        matrix transpose without touching a single byte on disk.
        Further, to reduce disk I/O, 'logicals' and non-standard data
        types get stored native and compact on binary flat files i.e.
        logicals take up exactly 2 bits to represent TRUE, FALSE and
        NA. Beyond basic access functions, the ff package also provides
        compatibility functions that facilitate writing code for ff and
        ram objects and support for batch processing on ff objects
        (e.g. as.ram, as.ff, ffapply). ff interfaces closely with
        functionality from package 'bit': chunked looping, fast bit
        operations and coercions between different objects that can
        store subscript information ('bit', 'bitwhich', ff 'boolean',
        ri range index, hi hybrid index). This allows to work
        interactively with selections of large datasets and quickly
        modify selection criteria. Further high-performance
        enhancements can be made available upon request.
License: file LICENSE
LazyLoad: yes
URL: http://ff.r-forge.r-project.org/
Encoding: latin1
Repository: CRAN
Repository/R-Forge/Project: ff
Repository/R-Forge/Revision: 38
Date/Publication: 2010-01-20 13:11:00
Packaged: 2010-01-17 21:07:55 UTC; rforge