Revision c9e32b8be95671340643c2e77a19af544af73266 authored by Ryan Lovelett on 12 January 2016, 22:07:27 UTC, committed by Ryan Lovelett on 13 January 2016, 01:56:33 UTC
Python 3 on Linux reads the system locale information to determine what it should use as the default encoding for strings read from files (this is different from OS X which is always UTF-8 by default [1]). Since all the Swift gyb templates are UTF-8 encoded there is effectively no reason to parse them as anything else. This patch forces the gyb template parser to read the template using UTF-8 encoding. It accounts for both reading and writing to a file as well as reading from stdin and writing to stdout. Two changes of note are that it now includes a __future__ import that should make Python 2 behave a little closer to Python 3 in terms of unicode support. Additionally Python 2 can no longer use cStringIO because it does not support unicode [2]. To test this patch I ran these commands before and after the patch. Note: that before the patch if the locale was set to something other than UTF-8, ASCII for instance, the Python 3 runs would fail. See [3] for example failure message. Without stdin/stdout: $ python2 utils/gyb -o Arrays.2.7.swift stdlib/public/core/Arrays.swift.gyb $ python3 utils/gyb -o Arrays.3.5.swift stdlib/public/core/Arrays.swift.gyb $ diff -u Arrays.2.7.swift Arrays.3.5.swift With stdin/stdout: $ cat stdlib/public/core/Arrays.swift.gyb | python2 utils/gyb > Arrays.2.7.stdin.stdout.swift $ cat stdlib/public/core/Arrays.swift.gyb | python3 utils/gyb > Arrays.3.5.stdin.stdout.swift $ diff -u Arrays.2.7.stdin.stdout.swift Arrays.3.5.stdin.stdout.swift [1] https://docs.python.org/3/howto/unicode.html#unicode-filenames [2] https://docs.python.org/2/library/stringio.html#cStringIO.StringIO [3] https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160111/000780.html
1 parent 7026909
.dir-locals.el
;===--- .dir-locals.el ---------------------------------------------------===;
;
; This source file is part of the Swift.org open source project
;
; Copyright (c) 2014 - 2016 Apple Inc. and the Swift project authors
; Licensed under Apache License v2.0 with Runtime Library Exception
;
; See http://swift.org/LICENSE.txt for license information
; See http://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
;
;===----------------------------------------------------------------------===;
;;; Directory Local Variables
;;; See Info node `(emacs) Directory Variables' for more information.
((nil
(tab-width . 2)
(eval .
;; Load the Swift project's settings. To suppress this action
;; you can put "(provide 'swift-project-settings)" in your
;; .emacs
(unless (featurep 'swift-project-settings)
;; Make sure the project's own utils directory is in the
;; load path, but don't override any one the user might have
;; set up.
(add-to-list
'load-path
(concat
(let ((dlff (dir-locals-find-file default-directory)))
(if (listp dlff) (car dlff) (file-name-directory dlff)))
"utils")
:append)
;; Load our project's settings -- indirectly brings in swift-mode
(require 'swift-project-settings)))
(c-file-style . "swift")
)
(c++-mode
(whitespace-style . (face lines indentation:space))
(eval . (whitespace-mode)))
(swift-mode
(whitespace-style . (face lines indentation:space))
(eval . (whitespace-mode)))
(objc-mode
(whitespace-style . (face lines indentation:space))
(eval . (whitespace-mode)))
(c-mode
(whitespace-style . (face lines indentation:space))
(eval . (whitespace-mode)))
(swift-mode
(tab-always-indent . t)))
;; Local Variables:
;; eval: (whitespace-mode -1)
;; End:
Computing file changes ...
