stringi package NEWS and CHANGELOG
===============================================================================

## Known bugs in ICU

* [REGEX] #147: regex look-behind assertions may fail to find a multibyte
Unicode search pattern [scheduled for ICU4C 56.1, see
http://bugs.icu-project.org/trac/ticket/11554]

-------------------------------------------------------------------------------

## 0.5-5 (2015-06-28) **CRAN**

* [BACKWARD INCOMPATIBILITY] `stri_install_check()` and `stri_install_icudt()`
are now deprecated. From now on they are supposed to be used only
by the `stringi` installer.

* [BUGFIX] #176: a patch for `sys/feature_tests.h` no longer included
(the original file was copyrighted by Sun Microsystems); fixed the *Compiler
or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX
applications* error by forcing (conditionally) `_XPG6` conformance.

* [BUGFIX] #174: `stri_paste()` did not generate any warning when
the recycling rule is violated and `sep==""`.

* [BUGFIX] #170: `icu::setDataDirectory` no longer called if our ICU source bundle
is not used (this used to cause build problems on openSUSE).

* [BUILD TIME] #169: `./configure` now tries to switch to the "standard"
C++ compiler if a C++11 one is not properly configured.

* [BUILD TIME] `configure.win` (`Biarch: TRUE`) now mimics `autoconf`'s
`AC_SUBST` and `AC_CONFIG_FILES` so that the build process is now
more similar across different platforms.

* [NEW FEATURE] `stri_info()` now also gives information on which ICU4C is
used (system or bundle).

-------------------------------------------------------------------------------

## 0.5-2 (2015-06-21) **CRAN**

* [BACKWARD INCOMPATIBILITY] The second argument to `stri_pad_*()` has
been renamed `width`.

* [GENERAL] #69: `stringi` is now bundled with ICU4C 55.1.

* [NEW FUNCTIONS] `stri_extract_*_boundaries()` extract text between text
boundaries.

* [NEW FUNCTION] #46: `stri_trans_char()` is a `stringi`-flavoured
`chartr()` equivalent.

* [NEW FUNCTION] #8: `stri_width()` approximates the *width* of a string
in a more Unicodish fashion than `nchar(..., "width")`

* [NEW FEATURE] #149: `stri_pad()` and `stri_wrap()` now by default bases on
code point widths instead of the number of code points. Moreover, the default
behavior of `stri_wrap()` is now such that it does not get rid
of non-breaking, zero width, etc. spaces

* [NEW FEATURE] #133: `stri_wrap()` silently allows for `width <= 0`
(for compatibility with `strwrap()`).

* [NEW FEATURE] #139: `stri_wrap()` gained a new argument: `whitespace_only`.

* [NEW FUNCTIONS] #137: date-time formatting/parsing:
     * `stri_timezone_list()` - lists all known time zone identifiers
     * `stri_timezone_set()`, `stri_timezone_get()` - manage current default time zone
     * `stri_timezone_info()` - basic information on a given time zone
     * `stri_datetime_symbols()` - localizable date-time formatting data
     * `stri_datetime_fstr()` - convert a `strptime`-like format string
            to an ICU date/time format string
     * `stri_datetime_format()` - convert date/time to string
     * `stri_datetime_parse()` - convert string to date/time object
     * `stri_datetime_create()` - construct date-time objects
            from numeric representations
     * `stri_datetime_now()` - return current date-time
     * `stri_datetime_fields()` - get values for date-time fields
     * `stri_datetime_add()` - add specific number of date-time units
            to a date-time object

* [GENERAL] #144: Performance improvements in handling ASCII strings
(these affect `stri_sub()`, `stri_locate()` and other string index-based
 operations)

* [GENERAL] #143: Searching for short fixed patterns (`stri_*_fixed()`) now
relies on the current `libC`'s implementation of `strchr()` and `strstr()`.
This is very fast e.g. on `glibc` utilizing the `SSE2/3/4` instruction set.

* [BUILD TIME] #141: a local copy of `icudt*.zip` may be used on package
install; see the `INSTALL` file for more information.

* [BUILD TIME] #165: the `./configure` option `--disable-icu-bundle`
forces the use of system ICU when building the package.

* [BUGFIX] locale specifiers are now normalized in a more intelligent way:
e.g. `@calendar=gregorian` expands to `DEFAULT_LOCALE@calendar=gregorian`.

* [BUGFIX] #134: `stri_extract_all_words()` did not accept `simplify=NA`.

* [BUGFIX] #132: incorrect behavior in `stri_locate_regex()` for matches
of zero lengths

* [BUGFIX] stringr/#73: `stri_wrap()` returned `CHARSXP` instead of `STRSXP`
on empty string input with `simplify=FALSE` argument.

* [BUGFIX] #164: `libicu-dev` usage used to fail on Ubuntu
(`LIBS` shall be passed after `LDFLAGS` and the list of `.o` files).

* [BUGFIX] #168: Build now fails if `icudt` is not available.

* [BUGFIX] #135: C++11 is now used by default (see the `INSTALL` file,
however) to build `stringi` from sources. This is because ICU4C uses the
`long long` type which is not part of the C++98 standard.

* [BUGFIX] #154: Dates and other objects with a custom class attribute
were not coerced to the character type correctly.

* [BUGFIX] Force ICU `u_init()` call on `stringi` dynlib load.

* [BUGFIX] #157: many overfull hboxes in the package PDF manual has been
corrected.

-------------------------------------------------------------------------------

## 0.4-1 (2014-12-11) **CRAN**

* [IMPORTANT CHANGE] `n_max` argument in `stri_split_*()` has been renamed `n`.

* [IMPORTANT CHANGE] `simplify=FALSE` in `stri_extract_all_*()` and
`stri_split_*()` now calls `stri_list2matrix()` with `fill=""`.
`fill=NA_character_` may be obtained by using `simplify=NA`.

* [IMPORTANT CHANGE, NEW FUNCTIONS] #120: `stri_extract_words` has been
renamed `stri_extract_all_words` and `stri_locate_boundaries` -
`stri_locate_all_boundaries` as well as `stri_locate_words` -
`stri_locate_all_words`. New functions are now available:
`stri_locate_first_boundaries`, `stri_locate_last_boundaries`,
`stri_locate_first_words`, `stri_locate_last_words`,
`stri_extract_first_words`, `stri_extract_last_words`.

* [IMPORTANT CHANGE] #111: `opts_regex`, `opts_collator`, `opts_fixed`, and
`opts_brkiter` can now be supplied individually via `...`.
In other words, you may now simply call e.g.
`stri_detect_regex(str, pattern, case_insensitive=TRUE)` instead of
`stri_detect_regex(str, pattern, opts_regex=stri_opts_regex(case_insensitive=TRUE))`.

* [NEW FEATURE] #110: Fixed pattern search engine's settings can
now be supplied via `opts_fixed` argument in `stri_*_fixed()`,
see `stri_opts_fixed()`. A simple (not suitable for natural language
processing) yet very fast `case_insensitive` pattern matching can be
performed now. `stri_extract_*_fixed` is again available.

* [NEW FEATURE] #23: `stri_extract_all_fixed`, `stri_count`, and
`stri_locate_all_fixed` may now also look for overlapping pattern
matches, see `?stri_opts_fixed`.

* [NEW FEATURE] #129: `stri_match_*_regex` gained a `cg_missing` argument.

* [NEW FEATURE] #117: `stri_extract_all_*()`, `stri_locate_all_*()`,
`stri_match_all_*()` gained a new argument: `omit_no_match`.
Setting it to `TRUE` makes these functions compatible with their
`stringr` equivalents.

* [NEW FEATURE] #118: `stri_wrap()` gained `indent`, `exdent`, `initial`,
and `prefix` arguments. Moreover Knuth's dynamic word wrapping algorithm
now assumes that the cost of printing the last line is zero, see #128.

* [NEW FEATURE] #122: `stri_subset()` gained an `omit_na` argument.

* [NEW FEATURE] `stri_list2matrix()` gained an `n_min` argument.

* [NEW FEATURE] #126: `stri_split()` now is also able to act
just like `stringr::str_split_fixed()`.

* [NEW FEATURE] #119:  `stri_split_boundaries()` now have
`n`, `tokens_only`, and `simplify` arguments. Additionally,
`stri_extract_all_words()` is now equipped with `simplify` arg.

* [NEW FEATURE] #116: `stri_paste()` gained a new argument:
`ignore_null`. Setting it to `TRUE` makes this function more compatible
with `paste()`.

* [OTHER] #123: `useDynLib` is used to speed up symbol look-up in
the compiled dynamic library.

* [BUGFIX] #114: `stri_paste()`: could return result in an incorrect order.

* [BUGFIX]  #94: Run-time errors on Solaris caused by setting
`-DU_DISABLE_RENAMING=1` -- memory allocation errors in i.a. ICU's
UnicodeString. This setting also caused some ABSan sanity check
failures within ICU code.

-------------------------------------------------------------------------------

## 0.3-1 (2014-11-06) **CRAN**

* [IMPORTANT CHANGE] #87: `%>%` overlapped with the pipe operator from
the `magrittr` package; now each operator like `%>%` has been renamed `%s>%`.

* [IMPORTANT CHANGE] #108: Now the BreakIterator (for text boundary analysis)
may be better controlled via `stri_opts_brkiter()` (see options `type`
and `locale` which aim to replace now-removed `boundary` and `locale` parameters
to `stri_locate_boundaries`, `stri_split_boundaries`, `stri_trans_totitle`,
`stri_extract_words`, `stri_locate_words`).

* [NEW FUNCTIONS] #109: `stri_count_boundaries()` and `stri_count_words()`
count the number of text boundaries in a string.

* [NEW FUNCTIONS] #41: `stri_startswith_*()` and `stri_endswith_*()`
determine whether a string starts or ends with a given pattern.

* [NEW FEATURE] #102: `stri_replace_all_*()` gained a `vectorize_all()` parameter,
which defaults to TRUE for backward compatibility.

* [NEW FUNCTION] #91: `stri_subset_*()`, a convenient and more efficient
substitute for `str[stri_detect_*(str, ...)]`, added.

* [NEW FEATURE] #100: `stri_split_fixed()`, `stri_split_charclass()`,
`stri_split_regex()`, `stri_split_coll()` gained a `tokens_only` parameter,
which defaults to `FALSE` for backward compatibility.

* [NEW FUNCTION] #105: `stri_list2matrix()` converts lists of atomic vectors
to character matrices, useful in connection with `stri_split`
and `stri_extract`.

* [NEW FEATURE] #107: `stri_split_*()` now allow setting an `omit_empty=NA` argument.

* [NEW FEATURE] #106: `stri_split()` and `stri_extract_all()` gained a `simplify`
argument (if `TRUE`, then `stri_list2matrix(..., byrow=TRUE)`
is called on the resulting list.

* [NEW FUNCTION] #77: `stri_rand_lipsum()` generates
(pseudo)random dummy *lorem ipsum* text.

* [NEW FEATURE] #98: `stri_trans_totitle()` gained a `opts_brkiter`
parameter; it indicates which ICU BreakIterator should be used when
performing case mapping.

* [NEW FEATURE] `stri_wrap()` gained a new parameter: `normalize`.

* [BUGFIX] #86: `stri_*_fixed()`, `stri_*_coll()`, and `stri_*_regex()` could
give incorrect results if one of search strings were of length 0.

* [BUGFIX] #99: `stri_replace_all()` did not use the `replacement` arg.

* [BUGFIX] #112: Some of the objects were not PROTECTed from
being garbage collected, which might have caused spontaneous SEGFAULTS.

* [BUGFIX] Some collator's options were not passed correctly to ICU services.

* [BUGFIX] Memory leaks causes as detected by
`valgrind --tool=memcheck --leak-check=full` have been removed.

* [DOCUMENTATION] Significant extensions/clean ups in the `stringi` manual.

-------------------------------------------------------------------------------

## 0.2-5 (2014-05-16) **CRAN**

* ~~icudt-dependent examples are no longer run if `icudt` is not available.~~

-------------------------------------------------------------------------------

## 0.2-4 (2014-05-15) **CRAN**

* [BUGFIX] Issues with loading of misaligned addresses in `stri_*_fixed()`.

-------------------------------------------------------------------------------

## 0.2-3 (2014-05-14) **CRAN**

* [IMPORTANT CHANGE] `stri_cmp*()` now do not allow for passing
 `opts_collator=NA`. From now on, `stri_cmp_eq`, `stri_cmp_neq`,
 and the new operators `%===%`, `%!==%`, `%stri===%`, and `%stri!==%`
 are locale-independent operations, which base on code point comparisons.
 New functions `stri_cmp_equiv` and `stri_cmp_nequiv`
 (and from now on also `%==%`, `%!=%`, `%stri==%`, and `%stri!=%`)
 test for canonical equivalence.

* [IMPORTANT CHANGE] `stri_*_fixed()` search functions now perform
 a locale-independent exact (byte-wise, of course after conversion to UTF-8)
 pattern search. All the Collator-based, locale-dependent search routines
 are now available via `stri_*_coll()`. The reason for this is that
 ICU USearch has currently very poor performance and in many search tasks
 in fact it is sufficient to do exact pattern matching.

* [GENERAL] `stri_*_fixed` now use a tweaked Knuth-Morris-Pratt search
 algorithm, which improves the search performance drastically.

* [IMPORTANT CHANGE] `stri_enc_nf*()` and `stri_enc_isnf*()` function families
have been renamed to `stri_trans_nf*()` and `stri_trans_isnf*()`,
respectively. This is because they deal with text transforming,
and not with character encoding. Moreover, all such operations may
also be performed by ICU's Transliterator (see below).

* [NEW FUNCTION] `stri_trans_general()` and `stri_trans_list()` give access
to ICU's Transliterator: may be used to perform very general
text transforms.

* [NEW FUNCTION `stri_split_boundaries()` utilizes ICU's BreakIterator
to split strings at specific text boundaries. Moreover,
stri_locate_boundaries indicates positions of these boundaries.

* [NEW FUNCTION] `stri_extract_words()` uses ICU's BreakIterator to
extract all words from a text. Additionally, `stri_locate_words`
locates start and end positions of words in a text.

* [NEW FUNCTION] `stri_pad()`, `stri_pad_left()`, `stri_pad_right()`,
and `stri_pad_both` pad a string with a specific code point.

* [NEW FUNCTION] `stri_wrap()` breaks paragraphs of text into lines.
Two algorithms (greedy and minimal raggedness) are available.

* [IMPORTANT CHANGE] `stri_*_charclass()` search functions now
rely solely on ICU's UnicodeSet patterns. All previously accepted
charclass identifiers became invalid. However, new patterns
should now be more familiar to the users (they are regex-like).
Moreover, we observe a very nice performance gain.

* [IMPORTANT CHANGE] `stri_sort()` now does not include `NA`s
in output vectors by default, for compatibility with `sort()`.
Moreover, currently none of the input vector's attributes are preserved.

* [NEW FUNCTION] `stri_unique()` extracts unique elements from
a character vector.

* [NEW FUNCTIONS] `stri_duplicated()` and `stri_duplicated_any()`
determine duplicate elements in a character vector.

* [NEW FUNCTION] `stri_replace_na()` replaces `NA`s in a character vector
with a given string, useful for emulating e.g. R's `paste()` behavior.

* [NEW FUNCTION] `stri_rand_shuffle()` generates a random permutation
of code points in a string.

* [NEW FUNCTION] `stri_rand_strings()` generates random strings.

* [NEW FUNCTIONS] New functions and binary operators for string comparison:
`stri_cmp_eq()`, `stri_cmp_neq()`, `stri_cmp_lt()`, `stri_cmp_le()`,
`stri_cmp_gt()`, `stri_cmp_ge()`, `%==%`, `%!=%`, `%<%`, `%<=%`, `%>%`, `%>=%`.

* [NEW FUNCTION] `stri_enc_mark()` reads declared encodings of character
strings as seen by `stringi`.

* [NEW FUNCTION] `stri_enc_tonative(str)` is an alias to
`stri_encode(str, NULL, NULL)`.

* [NEW FEATURE] `stri_order()` and `stri_sort()` now have an additional argument
`na_last` (defaults to `TRUE` and `NA`, respectively).

* [NEW FEATURE] `stri_replace_all_charclass()`, `stri_extract_all_charclass()`,
and `stri_locate_all_charclass()` now have a new arg, `merge`
(defaults to `FALSE` for backward-compatibility). It may be used
to e.g. replace sequences of white spaces with a single space.

* [NEW FEATURE] `stri_enc_toutf8()` now has a new `validate` arg (defaults
to FALSE for backward-compatibility). It may be used in a (rare) case
in which a user wants to fix an invalid UTF-8 byte sequence.
stri_length (among others) now detect invalid UTF-8 byte sequences.

* [NEW FEATURE] All binary operators `%???%` now also have aliases `%stri???%`.

* [GENERAL] Performance improvements in `StriContainerUTF8`
and `StriContainerUTF16` (they affect most other functions).

* [GENERAL] Significant performance improvements in `stri_join()`,
`stri_flatten()`, `stri_cmp()`, `stri_trans_to*()`, and others.

* [GENERAL] Added 3rd mirror site for our `icudt` binary distribution.

* `U_MISSING_RESOURCE_ERROR` message in `StriException` now suggests
calling `stri_install_check()`.

* [BUGFIX] UTF-8 BOMs are now silently removed from input strings.

* [BUGFIX] no more attempts to re-encode UTF-8 encoded strings
if native encoding=UTF-8 in `StriContainerUTF8`.

* [BUGFIX] possible memory leaks when throwing errors via `Rf_error()`.

* [BUGFIX] `stri_order()` and `stri_cmp()` could return incorrect results
for `opts_collator=NA`.

* [BUGFIX] `stri_sort()` did not guarantee to return strings in UTF-8.

-------------------------------------------------------------------------------

## 0.1-25 (2014-03-12) **CRAN**

* LICENSE tweaks.

* Initial CRAN release.

-------------------------------------------------------------------------------

## 0.1-24 (2014-03-11) **devel**

* Fixed bugs detected with `ASan` and `UBSan`,
   e.g. fixed `CharClass::gcmask` type (`enum` -> `uint32_t`)
   (reported by `UBSan`).

* Fixed array over-runs detected with `valgrind` in `string8.h`.

* Fixed unitialized class fields in `StriContainerUTF8`
   (reported by `valgrind`).

-------------------------------------------------------------------------------

## 0.1-23 (2014-03-11) **devel**

* License changed to BSD-3-clause, COPYRIGHTS updated.

* `icudt` is not shipped with `stringi` anymore;
   it is now downloaded in `install.libs.R` from one of our servers.

* New functions: `stri_install_check()`, `stri_install_icudt()`.

-------------------------------------------------------------------------------

## 0.1-22 (2014-02-20) **devel**

* System ICU is used on systems which do have one (version >= 50 needed).
  ICU is autodetected with `pkg-config` in `./configure`.
  Pass `'--disable-pkg-config'` to `./configure` to force building
  ICU from sources.

* `icudt52b` (custom subset) is now shipped with `stringi`
  (for big-endian, ASCII systems).

-------------------------------------------------------------------------------

## 0.1-21 (2014-02-19) **devel**

* Fixed some Solaris-related issues while preparing `stringi`
  for CRAN submission.

-------------------------------------------------------------------------------

## 0.1-20 (2014-02-17) **devel**

* ICU4C 52.1 sources included (common, i18n, stubdata + icu52dt.dat
  loaded dynamically). Compilation via Makevars.

* `stringi` now does not depend on any external libraries.

-------------------------------------------------------------------------------

## 0.1-11 (2013-11-16) **devel**

* ICU4C is now statically linked on Windows.

* First OS X binary build.

* The package is being intensively tested by our students @ FMIS WUT.

-------------------------------------------------------------------------------

## 0.1-10 (2013-11-13) **devel**

* Using `pkg-config` via `./configure` to look for ICU4C libs.

-------------------------------------------------------------------------------

## 0.1-6 (2013-07-05) **devel**

* First Windows binary build.

* Compilation passed on Oracle Sun Studio compiler collection.

* By now we have implemented most of the functionality
  scheduled for milestone 0.1.

-------------------------------------------------------------------------------

## 0.1-1 (2013-01-05) **devel**

* The `stringi` project has been established on GitHub.

-------------------------------------------------------------------------------