README.md
# Code and Data for [Rp] Reproducing: "Typographical features for scene text recognition"
Author: [Jerod Weinman](https://weinman.cs.grinnell.edu)
## Info
This archive is a collection of the scripts and libraries that
generate the experiments and data used in the following paper:
> Weinman, J. J. (2010, August). Typographical features for scene text
> recognition. In 2010 20th International Conference on Pattern
> Recognition (pp. 3987-3990). IEEE.
The data and code are archived for the purposes of documenting the
reproduction of the work as described in this paper:
> Weinman, J. (2020). [Rp] Reproducing: "Typographical features for
> scene text recognition". ReScience C. ISSN: 2430-3658.
## Licenses
Except where otherwise noted, the contents are authored by Jerod Weinman.
Except where other authorship is noted, all program code within is
licensed under the GNU General Public License (GPL) version 3.
Except where other authorship is noted, all text is published under a
[Creative Commons Attribution-Share Alike 4.0
License](https://creativecommons.org/licenses/by-sa/4.0/).
## Files
Although this repository contains unpacked versions of the code and
data (both within `data/`), the following list itemizes pre-packaged
subsets available (within `tar/`).
* **`collections.tar.bz2`**
Contains the raw source code for all the necessary experimental
collections in the data repository for the work. For more
information on the structure including supporting software tools,
see:
> Weinman, J. (2014). Data repository for reproducible research.
> Technical Report. Grinnell College.
> [DOI:11084/10001](https://doi.org/11084/10001).
* **`rawdata.tar.bz2`**
Contains the raw data collections used for the root of all
computations in the original and reproduction paper.
* **`cudamaxent.r231.tar.bz2`**
Contains the CUDA MaxEnt code documented as
being used in the original 2010 paper. For further details, see:
> Weinman, J. J., Lidaka, A., Aggarwal, S. (2011) Large-scale
> machine learning. In GPU Computing Gems Emeral Edition (Hwu,
> W. W., Ed.), pp. 277-291. Morgan Kaufman.
> [DOI:10.1016/B978-0-12-384988-5.00019-X](https://doi.org/10.1016/B978-0-12-384988-5.00019-X).
* **`cudamaxent.r284.tar.bz2`**
Contains the most recent CUDA MaxEnt code used as the basis of the
reproduction of results in the 2020 paper.
* **`cudamaxent.r284.diff`**
Contains the changes to the CUDA MaxEnt source code above for the
reproduction of results in the 2020 paper.
* **`cudamaxent.update.tar.bz2`**
Contains the updated CUDA MaxEnt code used for the reproduction of
results in the 2020 paper.
* **`vidi.r20.tar.bz2`**
Contains the parser and utility code documented as being used in the
original paper; also used in the reproduction. Note that the code
has been updated with bug fixes since the original publication.
* **`tools.r19.tar.bz2`**
Contains general Matlab tools documented as being used in the
original paper; also used in the reproduction. Note that the code
has been updated with bug fixes (particularly `lbfgs.m`) since the
original publication.
* **`pyrTools.tar.bz2`**
Contains the Wed Mar 28 2001 version of Eero Simoncelli's
matlabPyrTools Matlab toolbox used in the original paper and the
reproduction. Note that much newer versions have been made available
(though they are very similar), released under the MIT License.
## Dependencies
In addition to the standard build tools (e.g., `make`, `gcc`)
available on a Linux distribution (e.g., Ubuntu 18.04 LTS), the
following software is required:
* Matlab (e.g., R2018a), and its Image Processing, Statistics and
Machine Learning), Optimization, and Parallel Computing Toolboxes.
* CUDA (e.g., 10.1)
* Java (e.g., 1.8) and ant
## Install
Cloning the repository will generally give the packages in their
correct relative locations (under `data/`). The following instructions are for starting
from the raw packages listed above.
To install the software, the various packages need to be unpacked to
the approprite places.
1. Create a root for the data repository, set an environment
variable, and create the expected location for the library code
```sh
mkdir /data # Or a location of your choice
export DATA_REPOSITORY=/data
mkdir -p $DATA_REPOSITORY/svn/vision
```
2. Acquire and install the associated data repository skeleton and
tools from the archive at http://hdl.handle.net/11084/10024
(associated with DOI:11084/10001)
```sh
tar xf grinnell10001_5.tar -C $DATA_REPOSITORY/..
```
3. Unpack the various packages from this archive
```sh
tar xjf collections.tar.bz2 -C $DATA_REPOSITORY
tar xjf rawdata.tar.bz2 -C $DATA_REPOSITORY
tar xjf tools.r19.tar.bz2 -C $DATA_REPOSITORY/svn/vision
tar xjf vidi.r20.tar.bz2 -C $DATA_REPOSITORY/svn/vision
tar xjf cudamaxent.update.tar.bz2 -C $DATA_REPOSITORY/svn/vision
cp $DATA_REPOSITORY/svn/vision/cudamaxent/trunk/partrainc.m \
$DATA_REPOSITORY/svn/vision/tools/matlab/toolbox/recognition/@maxent/
tar xjf pyrTools.tar.bz2
mv pyrTools $DATA_REPOSITORY/svn/vision/tools/matlab/toolbox/pyramid
```
## Build
Although most of the code is Matlab, and therefore scripts that do not
need compilation, a few key pieces of software must be built.
When using a clone of the existing repository, the environment
variable `DATA_REPOSITORY` may be replaced with
`data/`.
1. Build the Java parsers for VIDI
```sh
cd $DATA_REPOSITORY/svn/vision/vidi/java
ant
```
2. Build the MEX files for VIDI and matlabPyrTools
```sh
cd $DATA_REPOSITORY/svn/vision/tools/matlab/toolbox/images/mex
MATLAB_ROOT/bin/mex boxfilter.c
cd $DATA_REPOSITORY/svn/vision/tools/matlab/toolbox/pyramid
make # The Makefile may need adjustment to your platform
```
3. Build CUDA MaxEnt
```sh
cd $DATA_REPOSITORY/svn/vision/cudamaxent
make cuda_value.mexa64 # Makefile may need adjustment to your platform
```
## Run
The collections may be run in a breadth-first traversal of their
dependency graph---see Figure 1 in Weinman (2020):
To run a collection, enter the directory and invoke `make` (no targets
should be given).
Each collection may need some local customization to the platform. In
particular, the `pathdef.m` would more easily be replaced with an edit
to `startup.m` that adds the Matlab tools directories, i.e.,
```matlab
addpath(genpath(fullfile(getenv('DATA_REPOSITORY'),'svn/vision/tools')));
```
## Collections Key
The following key maps the coded experimental collections listed in
the paper (Weinman, 2020) to the actual paths in `collections.tar.bz2`
(beneath `data/` in this repository).
| Collection | Path |
| ---------- | ---- |
| eA | `experiments/scenetext/parsing/recognition/widths_spaces_gaps/context_borders_contrast_bias_noise_widths/baseline/bigram/binary_maxent_whiten_R_vary_chPr_82em2_fix_I_spaces-20100101124727` |
| eB | `experiments/text/ngrams/bigrams/tied_nums_intracase_L1_validation-20090708075734` |
| eC | `experiments/scenetext/parsing/recognition/widths_spaces_gaps/context_borders_contrast_bias_noise_widths/baseline/overlap/binary_maxent_whiten_R_vary_chPr_82em2_fix_I_spaces-20100101114951` |
| eD | `experiments/letters/recognition/widths_spaces_gaps/context_borders_contrast_bias_noise_widths/binary_maxent_whiten_R_vary_chPr_82em2_fix_I-20091029090444` |
| eE | `experiments/text/meta/corpus_ten_fold_split-20090708080932` |
| eF | `experiments/scenetext/parsing/recognition/widths_spaces_gaps/context_borders_contrast_bias_noise_widths/baseline/width/binary_maxent_whiten_R_vary_chPr_82em2_fix_I_spaces-20100101153837` |
| eG | `experiments/letters/meta/width_prediction_spaces_noextrema-20100101111128` |
| eH | `experiments/letters/meta/width_scatterplot-20090618153700` |
| eI | `experiments/letters/meta/width_prediction-20090625071311` |
| eJ | `experiments/scenetext/parsing/recognition/widths_spaces_gaps/context_borders_contrast_bias_noise_widths/baseline/appearance/binary_maxent_whiten_R_vary_chPr_82em2_fix_I_spaces-20100101114313` |
| pA | ``proc/gaps/images/context_borders_contrast_bias_window_128-20090727135842`` |
| pB | `proc/gaps/images/context_borders_window_128-20090727135259` |
| pC | `proc/letters/images/context_borders_widths_window_128-20091013083000` |
| pD | `proc/text/bigrams-20090618132847` |
| pE | `proc/letters/images/context_borders_contrast_bias_window_128-20090714081648` |
| pF | `proc/letters/images/context_borders_window_128-20090714075726` |
| pG | `proc/gaps/filtered/context_borders_contrast_bias_noise_pyramid_6_rectified_normalized_window_32-20090727141649` |
| pH | `proc/gaps/images/context_borders_contrast_bias_noise_window_32-20090727140344` |
| pI | `proc/letters/images/context_borders_binary_widths_window_32-20091015093132` |
| pJ | `proc/letters/filtered/context_borders_contrast_bias_noise_pyramid_6_rectified_normalized_window_32-20090714100804` |
| pK | `proc/spaces/images/context_borders_binary_window_32-20090728144643` |
| pL | `proc/spaces/filtered/context_borders_contrast_bias_noise_pyramid_6_rectified_normalized_window_32-20090714142320` |
| pM | `proc/letters/images/context_borders_binary_window_32-20090728143416` |
| pN | `proc/letters/filtered/context_borders_contrast_bias_noise_pyramid_6_rectified_normalized_widths_window_32-20091015081437` |
| pO | `proc/gaps/images/context_borders_binary_window_32-20090728145237` |
| pP | `proc/letters/images/context_borders_contrast_bias_noise_widths_window_32-20091014171150` |
| pQ | `proc/letters/images/context_borders_contrast_bias_widths_window_128-20091013093715` |
| pR | `proc/letters/images/context_borders_contrast_bias_noise_window_32-20090714090438` |
| pS | `proc/letters/meta/widths-20090618151515` |
| pT | `proc/spaces/images/context_borders_contrast_bias_noise_window_32-20090714141559` |
| pU | `proc/spaces/images/context_borders_contrast_bias_window_128-20090714134138` |
| pV | `proc/spaces/images/context_borders_window_128-20090714133605` |
| rA | `raw/scenetext/reading/sign_info-20051118114032` |
| rB | `raw/scenetext/reading/sign_images-20030912134700` |
| rC | `raw/letters/normal-20050805083000` |
| rD | `raw/text/books` |
