Revision c9e32b8be95671340643c2e77a19af544af73266 authored by Ryan Lovelett on 12 January 2016, 22:07:27 UTC, committed by Ryan Lovelett on 13 January 2016, 01:56:33 UTC
Python 3 on Linux reads the system locale information to determine what it should use as the default encoding for strings read from files (this is different from OS X which is always UTF-8 by default [1]). Since all the Swift gyb templates are UTF-8 encoded there is effectively no reason to parse them as anything else. This patch forces the gyb template parser to read the template using UTF-8 encoding. It accounts for both reading and writing to a file as well as reading from stdin and writing to stdout. Two changes of note are that it now includes a __future__ import that should make Python 2 behave a little closer to Python 3 in terms of unicode support. Additionally Python 2 can no longer use cStringIO because it does not support unicode [2]. To test this patch I ran these commands before and after the patch. Note: that before the patch if the locale was set to something other than UTF-8, ASCII for instance, the Python 3 runs would fail. See [3] for example failure message. Without stdin/stdout: $ python2 utils/gyb -o Arrays.2.7.swift stdlib/public/core/Arrays.swift.gyb $ python3 utils/gyb -o Arrays.3.5.swift stdlib/public/core/Arrays.swift.gyb $ diff -u Arrays.2.7.swift Arrays.3.5.swift With stdin/stdout: $ cat stdlib/public/core/Arrays.swift.gyb | python2 utils/gyb > Arrays.2.7.stdin.stdout.swift $ cat stdlib/public/core/Arrays.swift.gyb | python3 utils/gyb > Arrays.3.5.stdin.stdout.swift $ diff -u Arrays.2.7.stdin.stdout.swift Arrays.3.5.stdin.stdout.swift [1] https://docs.python.org/3/howto/unicode.html#unicode-filenames [2] https://docs.python.org/2/library/stringio.html#cStringIO.StringIO [3] https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160111/000780.html
1 parent 7026909
File | Mode | Size |
---|---|---|
_static | ||
_templates | ||
archive | ||
proposals | ||
scripts | ||
tools | ||
ABI.rst | -rw-r--r-- | 56.1 KB |
ARCOptimization.rst | -rw-r--r-- | 21.2 KB |
AccessControl.rst | -rw-r--r-- | 10.9 KB |
AccessControlInStdlib.rst | -rw-r--r-- | 3.4 KB |
Array.png | -rw-r--r-- | 68.3 KB |
ArrayBridge.graffle | -rw-r--r-- | 42.3 KB |
ArrayBridge.png | -rw-r--r-- | 160.3 KB |
ArrayCast.graffle | -rw-r--r-- | 38.9 KB |
ArrayCast.png | -rw-r--r-- | 125.0 KB |
ArrayImplementation.graffle | -rw-r--r-- | 34.8 KB |
ArrayImplementation.png | -rw-r--r-- | 272.6 KB |
Arrays.rst | -rw-r--r-- | 10.0 KB |
CMakeLists.txt | -rw-r--r-- | 5.1 KB |
CallingConvention.rst | -rw-r--r-- | 56.6 KB |
ContiguousArray.png | -rw-r--r-- | 39.0 KB |
DebuggingTheCompiler.rst | -rw-r--r-- | 10.3 KB |
Dependency Analysis.rst | -rw-r--r-- | 5.0 KB |
DocumentationComments.md | -rw-r--r-- | 2.8 KB |
DriverInternals.rst | -rw-r--r-- | 5.5 KB |
DriverParseableOutput.rst | -rw-r--r-- | 4.9 KB |
ErrorHandling.rst | -rw-r--r-- | 28.7 KB |
ErrorHandlingRationale.rst | -rw-r--r-- | 89.8 KB |
Failable Initializers.rst | -rw-r--r-- | 8.7 KB |
GenerateTopLevelLitreCMakeLists.cmake | -rw-r--r-- | 206 bytes |
Generics.rst | -rw-r--r-- | 35.5 KB |
GitWorkflows.rst | -rw-r--r-- | 9.5 KB |
HighLevelSILOptimizations.rst | -rw-r--r-- | 14.3 KB |
Import.rst | -rw-r--r-- | 4.7 KB |
IndexInvalidation.rst | -rw-r--r-- | 7.9 KB |
Lexicon.rst | -rw-r--r-- | 8.4 KB |
LibraryEvolution.rst | -rw-r--r-- | 45.2 KB |
Literals.rst | -rw-r--r-- | 5.5 KB |
LitreTemplate.cmake | -rw-r--r-- | 499 bytes |
LogicalObjects.rst | -rw-r--r-- | 4.8 KB |
Makefile | -rw-r--r-- | 5.4 KB |
Modules.rst | -rw-r--r-- | 17.3 KB |
MutationModel.rst | -rw-r--r-- | 9.5 KB |
ObjectInitialization.rst | -rw-r--r-- | 20.1 KB |
OptimizationTips.rst | -rw-r--r-- | 20.4 KB |
OptimizerDesign.md | -rw-r--r-- | 10.8 KB |
Pattern Matching.rst | -rw-r--r-- | 32.0 KB |
Runtime.md | -rw-r--r-- | 14.2 KB |
SIL.rst | -rw-r--r-- | 157.1 KB |
SequencesAndCollections.rst | -rw-r--r-- | 13.5 KB |
Serialization.rst | -rw-r--r-- | 7.1 KB |
Slice.png | -rw-r--r-- | 86.4 KB |
StdlibAPIGuidelines.rst | -rw-r--r-- | 7.3 KB |
StdlibRationales.rst | -rw-r--r-- | 10.7 KB |
StoredAndComputedVariables.rst | -rw-r--r-- | 7.7 KB |
StringDesign.rst | -rw-r--r-- | 53.1 KB |
Testing.rst | -rw-r--r-- | 15.1 KB |
TextFormatting.rst | -rw-r--r-- | 15.4 KB |
TransparentAttr.rst | -rw-r--r-- | 3.4 KB |
TypeChecker.rst | -rw-r--r-- | 46.4 KB |
conf.py | -rw-r--r-- | 8.8 KB |
contents.rst | -rw-r--r-- | 372 bytes |
doxygen.cfg.in | -rw-r--r-- | 50.8 KB |
doxygen.css | -rw-r--r-- | 9.0 KB |
doxygen.footer | -rw-r--r-- | 378 bytes |
doxygen.header | -rw-r--r-- | 475 bytes |
doxygen.intro | -rw-r--r-- | 873 bytes |
favicon.ico | -rw-r--r-- | 0 bytes |
toc.js | -rw-r--r-- | 4.2 KB |
weak.rst | -rw-r--r-- | 50.0 KB |
Computing file changes ...