https://github.com/JuliaLang/julia

sort by:
Revision Author Date Message Commit Date
07d5785 WIP 11 September 2019, 21:14:24 UTC
ef948b5 Fix a couple bugs 05 August 2019, 23:18:29 UTC
e703891 Very WIP: [wasm] Bidirectional Julia/JS object access This is a start at what I think is the last major feature on the wasm porting punch list: sharing/accessing objects between Julia and the native javascript environment. This PR currently contains a rough sketch of the interpreter version of this integration, though we should of course try to get the compiled version going as soon as possible, since we now ship a compiled system image for the wasm target. For the compiled version of this we'd like to use the reference-types (https://github.com/WebAssembly/reference-types) wasm extension, which is currently in development, so even the interpreter parts are written with that extension in mind. In particular, we have a wasm table of javascript objects that represents our (indexed) managed heap of javascript objects. A boxed reference to javascript is then the julia type tag, plus an index into this table (in compiled mode an unboxed reference would just be a raw `anyref` without going through the table). The GC is responsible for managing this table of javascript objects, marking any JS objects that are reachable and sweeping the unreachable ones by nulling out the references (this part isn't implemented yet). On the julia side, we provide one julia type for every javascript builtin type ('number' corresponds to Float64 and 'boolean' to Bool, but otherwise the types are dedicated), with the non-singleton types being primitive types of pointer size to store the index into the table of javascript objects (at least semantically, as mentioned codegen will not use the table if possible). There is a bit of logic in javascript to convert these boxed representations on the julia heap into proper javascript objects, but otherwise the julia code is responsible for conversions from Julia objects to javascript objects (e.g. `String` to `JSString`, or `Ptr` to `JSNumber` - i.e. Float64). A `@jscall` macro is provided to call javascript functions from within julia. Under the hood this lowers to an `Expr(:foreigncall, ...)` with `jscall` calling convention for which we implement support in the interpreter. I don't forsee any problems with codegen for this representation, but that part is not implemented yet. For the reverse (referencing julia objects from javascript), I mostly followed what pyodide does and used a javascript `Proxy` object that wraps a boxed julia pointer (set/getproperty aren't implemented yet, but that should be straightforward). For GC integration, we make use of the (experimental) [JavaScript WeakRef](https://github.com/tc39/proposal-weakrefs) support, which essentially implements finalizers. On the julia side, I stole one of the remaining GC bits (at least one of the bits documented to be remaining - it looks like we don't overalign types sufficiently for this to actually work at the moment) to mark an object as being borrowed by the external VM and thus being protected from collection even if there is no remaining julia references. The JS side finalizer clears this bit on an object when there are no remaining references on the JS side. As always with these kinds of systems, reference cycles are not collected. I don't think there's much to be done about that, until WebAssembly gets more native integration with the JavaScript JC and we can re-use that for Julia objects. It should be noted that the WeakRef proposal is curently behind a flag in both Firefox and Chrome, but my understanding is that the timeline for this fetaure is comparable to that of the reference types proposal this builds on, so it should be ok to use (and I don't know of any other mechanism to learn whether the JS side has been collected). Remaining TODOs: - [ ] Hook up get/setproperty on the JS side - [ ] Julia GC marking/sweeping slots in the JS object table - [ ] Codegen For codegen, the major obstacle is lack of support in the toolchains (llvm, emscripten). I'm thinking it should be possible to just represent anyref as a non-integral pointer type in LLVM and get somewhere, but plumbing everything through is still a decent amount of work, that I'm unlikely to have the time for - I'm hoping somebody else will take that on. 05 August 2019, 04:26:33 UTC
837f5e0 Don't init load path on wasm, we don't have a file system 31 July 2019, 02:06:00 UTC
8863095 Regenerate foreigncall 31 July 2019, 02:05:01 UTC
ae83be7 Move displaysize to io.jl 31 July 2019, 01:34:40 UTC
6fe3c5b Fix pagetable1 sweeping on 32bit If no pool allocations have occurred before the first GC (e.g. because MEMDEBUG was set), the pagetable1 that covers the entire (32bit) memory region on 32 bit machines may not have been initialized yet, while the corresponding sweep function was making the assumption that the passed in pagetable was non-null. Add the appropriate check for initialization. 30 July 2019, 21:06:39 UTC
15f9e95 Fix mpfr calls 29 July 2019, 08:13:57 UTC
2a0b52d setjmp is important for the wasm abi 29 July 2019, 07:54:07 UTC
8e247aa Add a helpful assertion 29 July 2019, 07:43:32 UTC
9ec1525 Move rngseed 29 July 2019, 07:37:50 UTC
04cad58 Fix the signature thing 29 July 2019, 06:45:46 UTC
f5cc3ed WIP 29 July 2019, 06:39:44 UTC
e357592 WIP 29 July 2019, 03:06:58 UTC
4e96357 Cleanup 17 July 2019, 18:43:17 UTC
db1ee7b Disable COPY_STACKS on wasm This doesn't really make sense on wasm (or perhaps there's no non-COPY_STACKS mode since we're always unwinding and rewinding). 11 July 2019, 19:35:08 UTC
f747291 Tasking for Emscripten/Wasm target This is an implementation of Julia's coroutine/tasking system on top of the binaryen Bysyncify transform [1] recently implemented by @kripken. The wasm target is unusual in several ways: 1. It has an implicitly managed call stack that we may not modify directly (i.e. is only modified through calls/returns). 2. The event loop is inverted, in that the browser runs the main event loop and we get callbacks for events, rather than Julia being the main event loop. Bysyncify takes care of the first problem by providing a mechanism to explicitly unwind and rewind the implicitly managed stack (essentially copying it to an explicitly managed stack - see the linked implementation for more information). For the second, I am currently using the ptls root_task to represent the browser event loop, i.e. yielding to that task will return control back to the browser and this is the task in which functions called by javascript will run unless they explicitly construct a task to run. As a result, julia code executed in the main task may not perform any blocking operations (though this is currently not enforced). I think this is a sensible setup since the main task will want to run some minor julia code (e.g. to introspect some data structures), but the bulk of the code will run in their own tasks (e.g. the REPL backend task). [1] https://github.com/WebAssembly/binaryen/blob/master/src/passes/Bysyncify.cpp 11 July 2019, 19:29:01 UTC
3813dc8 Make exit more forceful 11 July 2019, 19:26:41 UTC
1c34350 Regenerate interpreter 11 July 2019, 19:26:41 UTC
55b1169 Fix missing GC root 11 July 2019, 19:26:41 UTC
afe2e0b Fix arg type of jl_switchto 11 July 2019, 19:26:41 UTC
e32d1f3 Fix number of string args 11 July 2019, 19:26:41 UTC
a58f415 Fix all the things after rebase 11 July 2019, 19:26:41 UTC
256c8df WIP 11 July 2019, 19:26:41 UTC
b6a5642 add JSON 11 July 2019, 19:26:41 UTC
2bf4f47 add PlotlyBase 11 July 2019, 19:26:41 UTC
c6f7aa2 add PlotlyBase 11 July 2019, 19:26:41 UTC
8b9b7ed Add a fallback mechanism for loading packages 11 July 2019, 19:26:41 UTC
e202339 Regenerate interpreter 11 July 2019, 19:26:41 UTC
022591e Add a utility to instantiate the foreigncall rt 11 July 2019, 19:26:41 UTC
5a781cc fixes for Random 11 July 2019, 19:26:41 UTC
b7c9b03 add back Random 11 July 2019, 19:26:41 UTC
37f798e fake threadding constructs 11 July 2019, 19:26:41 UTC
95d3ecb Regenerate interpreter 11 July 2019, 19:26:41 UTC
95cf3d2 Regenerate interpreter 11 July 2019, 19:26:41 UTC
c36abbc Regenerate interpreter 11 July 2019, 19:26:41 UTC
38de8ad Regenerate interpreter 11 July 2019, 19:26:41 UTC
7ba4e14 Regenerate interpreter 11 July 2019, 19:26:41 UTC
33735f7 Regenerate interpreter 11 July 2019, 19:26:41 UTC
624b319 Check in latest generated foreigncall interpreter 11 July 2019, 19:26:41 UTC
e28e8d2 Bugfix 11 July 2019, 19:26:41 UTC
a1795d1 Make interpreter-foreigncall optional 11 July 2019, 19:26:41 UTC
70aeaac Don't disable MPFR in emscripten build 11 July 2019, 19:26:41 UTC
425e752 Switch to autogenerated foreigncall in interpreter 11 July 2019, 19:26:41 UTC
0626f9b Don't map pcre/gmp for native build 11 July 2019, 19:26:41 UTC
3643137 Get farther 11 July 2019, 19:26:41 UTC
f9c1689 Allow overriding the build arch/os constants 11 July 2019, 19:26:41 UTC
aeac955 Wrap GMP 11 July 2019, 19:26:41 UTC
83b544b Finish mapping PCRE 11 July 2019, 19:26:41 UTC
56c364c Add mapping for PCRE calls 11 July 2019, 19:26:41 UTC
780975f HACK: Manually wrap some common foreign calls in interpreter 11 July 2019, 19:26:41 UTC
22d64be HACK: Force emscripten to use the interpreter 11 July 2019, 19:26:41 UTC
f71410e dont cache since jl_return_address not available 11 July 2019, 19:26:41 UTC
99a20f4 Disable various unsupported startup things on emscripten 11 July 2019, 19:26:41 UTC
29cf12f ui fixup 11 July 2019, 19:26:41 UTC
8d227f1 Add ui/ file for wasm 11 July 2019, 19:26:41 UTC
6540c52 Error when trying to start a fiber on wasm 11 July 2019, 19:26:41 UTC
dec05b8 Emscripten build fixes 11 July 2019, 19:26:41 UTC
224a261 Disable dependencies that don't currently build for emscripten 11 July 2019, 19:26:41 UTC
fa178a1 Add an option to build without libuv There is increasing interest in running Julia in environments where libuv is not available (JSVM, baremetal library OSes, etc). This adds a make file option to disable building libuv entirely and fixes up the bootstrap process to go through despite libuv not being available. There is of course a question of whether instead of this it makes sense to just port libuv to those platforms, but I suspect the answer is no, because the most core part of libuv (async sockets, the event loop, spawning processes), must either have a completely different implementation or is unavailable entirely. Particularly for JSVM it is also unlikely that libuv upstream is particularly interested, because JavaScript already runs natively in JSVM. One slightly weird part is that these platforms do generally implement POSIX where possible, so in a number of instances, I had to replace the libuv calls (which wrap POSIX calls on non-Windows platforms) by the corresponding POSIX calls. Long term I see a few options: - Expand libuv support for these new platforms - Seperate libuv into two libraries, one for async IO, process spawning, event loop etc, and one for the POSIX-compat layer on Windows. - Use POSIX calls everywhere on platforms that support it and use libuv for windows support where necessary (since windows does also implement POSIX for some things). However, until then, this build flag is useful as a basis for rebuilding these abstraction on top of the JSVM APIs and for simplifying boostrap on other platforms. However, I don't envision this to be a supported or tested configuration on platforms where libuv is otherwise available. 11 July 2019, 19:26:41 UTC
24efaa4 Fix various alignment violations (#32513) The C standard forbids accessing pointers that are not sufficiently aligned for their base type under the penalty of undefined behavior. On most architectures we support, this doesn't make a difference (except if the compiler tries to SIMD things), but it does matter on some (e.g. SAFE_HEAP mode in wasm), so make some progress on using the proper unaligned accessors in various places that use underaligned pointers. 11 July 2019, 19:19:56 UTC
c690489 io: corrected error handling for streams (#32429) After #32309, we became much more eager to forget about normal errors and more eager to throw an EOFError at the wrong place. This is intended to fix that oversight. 11 July 2019, 19:11:17 UTC
1a37227 thread safety fix in compiling regexes (#32544) 11 July 2019, 18:29:13 UTC
f5a50be Future doc (#31737) 10 July 2019, 15:11:31 UTC
073cbbb Add empty(::NamedTuple) (#32534) 10 July 2019, 01:50:27 UTC
13db0dc partr: fix rare segfault (#32530) The write to 'prio' might be arbitrarily delayed, after some other thread has already popped the task. This commit ensures that all writes to that field are always behind the same lock as the modification to the queue itself. fix #32520 09 July 2019, 20:32:41 UTC
f18808d doc: fix typo in sort! docstring (#32533) 09 July 2019, 08:56:09 UTC
f552fb2 Add methods min, max, minmax taking only single Dates.AbstractTime (#32525) Methods `min`, `max` and `minmax` should also be able to take just one `Dates.AbtractTime` argument. See discussion: https://discourse.julialang.org/t/max-and-min-of-single-dates/26074 08 July 2019, 18:17:40 UTC
33a6b99 rational: `signbit(den)` instead of `sign(den) < 0` (#32523) 08 July 2019, 15:25:49 UTC
a25e722 fix #32499, regression in struct expr in assignment rhs (#32512) caused by c6c3d72d1cbddb3d27e0df0e739bb27dd709a413 This is a bit of a hack that just moves things around a bit to get a structure more likely to work. 08 July 2019, 10:03:05 UTC
d611e1c docs: Change `1:size(A, i)` to `axes(A, i)` in at-nloops docs (#32519) 08 July 2019, 06:12:20 UTC
c351391 Yet another LLVM patch (#32506) The previous iteration of this wasn't quite right and I didn't catch it before merging it in julia (or upstream for that matter), so here's the fixup patch. 07 July 2019, 20:13:54 UTC
89baf83 added *larf and *larfg LAPACK wrappers (#32390) * added LAPACK wrappers for *larf and *larfg routines * add fixes suggested by vtjnash 06 July 2019, 21:10:43 UTC
a9ad6c2 fix #32488, type intersection regression (#32509) Issue was caused by 74305bf2c43546bf76843b72ff0d268a3fe16920, so this reverts part of the new logic from there. 06 July 2019, 15:55:01 UTC
1272185 fix and tests (#32328) 06 July 2019, 15:47:48 UTC
1261a25 make getindex O(1) worst-case (#32329) 06 July 2019, 15:47:22 UTC
82f1fae Use tempdir in tempname and update tempdir warning in docstring (#32424) 06 July 2019, 15:20:30 UTC
a060344 making Bi/Tri/Sym times Diag output a structured matrix (not sparse) (#31889) * making Bi/Tri/Sym times Diag output a structured matrix (not sparse) * fix test errors/typos * adding diag*bi/tri/sym and removing unused union * add mul!(C,::BiTriSym, ::*Diagonal) * fixing ambiguious methods and adjoint/transpose * space * test error changing `transpose`/`adjoint` to `Transpose`/`Adjoint` This will fail until https://github.com/JuliaLang/julia/pull/31889 is merged. * remove transpose 06 July 2019, 05:54:05 UTC
e6379da Export istaskfailed (#32300) * Export istaskfailed * Add docstring for istaskfailed * Add news for istaskfailed 05 July 2019, 19:41:06 UTC
b2ac7a3 Fix #30944, ranges with non-IEEEFloat types (#30952) 05 July 2019, 19:08:21 UTC
cd79fe6 speed up filter of vector (#31929) 05 July 2019, 18:38:19 UTC
95ebe8f Document finalizer task-switch restriction (#25141) 05 July 2019, 18:33:27 UTC
2350c04 fix #32465, inference of splatting unionall NamedTuple types (#32494) 05 July 2019, 16:14:01 UTC
8792eb2 Merge pull request #31928 from Moelf/date_name_doc Document Dates.*name() integer method 05 July 2019, 15:20:48 UTC
729e87f more performant SparseMatricCSC constructors for structured matrices (#32492) 05 July 2019, 08:51:49 UTC
0060e13 More Makefile definitions for targeting emscripten (#32482) 04 July 2019, 21:29:32 UTC
4d68846 Fix tab completion in Logging for functions from CoreLogging (#32473) CoreLogging is an implementation detail, but the fact that Logging imports things from there breaks tab completion for the parts of the public API which aren't re-exported. Fix this by explicitly creating bindings for each imported symbol. 04 July 2019, 11:49:58 UTC
3d41f27 Upgrade Pkg to v1.2.0-rc3. (#32487) 04 July 2019, 07:47:08 UTC
61a0b04 Update ssair.md (#32489) fix typo 03 July 2019, 20:58:54 UTC
a63f2e9 Merge pull request #32485 from JuliaLang/kf/rettypert Switch jl_rettype_inferred sentintel to jl_nothing not NULL 03 July 2019, 20:54:27 UTC
4e1dd9d improve printing of method parameter bounds in candidate list (#32484) 03 July 2019, 20:43:28 UTC
38b38cf Address part of #28866, prevent array operations from dropping 0d containers (#32122) This is a simple workaround for the handful of elementwise operations that are defined on arrays _without_ the need for explicit broadcast but use broadcasting (with an extra shape check) in their implementation. These were the only affected cases I could find. 03 July 2019, 19:30:34 UTC
b74ef05 fix #32467, parsing macro on `for` inside `do` inside call (#32476) 03 July 2019, 16:48:08 UTC
a17d9ae SparseMatrixCSC constructors for structured matrices - Diagonal, Tridiagonal, etc. (#32466) * move "sparse" code for structured matrices to "SparseMatrixCSC" * spaces between arguments * don't use fallback sparse(::AbstractMatrix) for structured matrices * add tests * fix typo in test 03 July 2019, 14:11:27 UTC
c935e13 Update `-h` flag documentation in getting-started (#32480) 03 July 2019, 14:00:32 UTC
28724b1 filter specialization for Strings (#32460) (#32478) 03 July 2019, 13:10:51 UTC
4b6ab68 bug fixed in read_to_buffer (#32457) * bug fixed in read_to_buffer * added test for #32397. * switch back to original case * added torture test for long encoded string randomly laced with spaced 03 July 2019, 06:45:53 UTC
07af3d2 Merge pull request #32481 from JuliaLang/kf/typofix Small typo fix 03 July 2019, 04:21:55 UTC
9783c91 specialize matrix multiplication with Diagonals (#32362) * specialize matrix multiplication with Diagonals * use broadcasting everywhere 03 July 2019, 01:26:01 UTC
ef89587 Switch jl_rettype_inferred sentintel to jl_nothing not NULL Using NULL requires treating it as a raw pointer and doing an unsafe operation to recover the object reference. We generally try to avoid NULL sentintels in such situations, using nothing instead. Do the same here. 03 July 2019, 00:44:59 UTC
back to top