https://github.com/JuliaLang/julia

sort by:
Revision Author Date Message Commit Date
9b8e575 fall through 02 December 2023, 20:33:33 UTC
f2df9d3 Merge branch 'master' into sb/sigfpe2 02 December 2023, 20:01:32 UTC
641f717 make custom log macros work (#52359) 02 December 2023, 14:36:18 UTC
aef528d nfc: remove duplicate variable declaration (#52360) 01 December 2023, 03:51:11 UTC
58c1d51 [REPLCompletions] fix #52099 by adjusting effects of `HAMT` methods (#52331) After looking into #52099, I discovered that it has the same root cause as #51548. Essentially, when a method that's not `!effect_free` is applied to a `Const` value concretized by the `REPLInterpreter`'s aggressive inference, since the `!effect_free` method will not be concretized, it will eventually lead to the `Const` representing unexpected object being returned. This commit tries to fix the specific problem reported in #52099 by enhancing the effects of `Base.HAMT` methods, so they're concrete-evaled. Admittedly, this is more of a quick fix than a complete solution, and not the best one, but it was the simplest. A better solution might involve implementing EA's handling of `Memory`-objects that allows the compiler to automatically prove `:effect_free` in more scenarios. But this would need a bigger overhaul, so I plan to tackle it in another PR. 01 December 2023, 00:12:35 UTC
bac3ba5 Fix oracle count violation in sroa_pass! (#52336) 30 November 2023, 03:12:47 UTC
fd67cb2 incorporate upstream fixes to crc32c.c assembly (#52326) These are a response to [this comment](https://stackoverflow.com/questions/17645167/implementing-sse-4-2s-crc32c-in-software/17646775#comment88832559_17646775) on StackOverflow: > Your asm constraints aren't strictly safe. A pointer in a `"r"` constraint does not imply that the pointed-to memory is also an input. You could just use memory operands (or the `_mm_crc32_u64` intrinsic), but if you want to force the addressing mode you can use a dummy memory operand like `asm("crc32 (%1),%0 " : "+r"(crc0) : "r"(next), "m" (*(const char (*)[24]) pStr) )`;. Inlining or LTO could break this. See [at&t asm inline c++ problem](https://stackoverflow.com/posts/comments/81680441) (which needs an update: pointer-to-array is cleaner than a struct with a flexible array member). The existing test coverage wasn't enough to fully exercise Adler's code, so I added another test. Closes #52325 29 November 2023, 21:13:04 UTC
9bd2432 cfg_simplify: Fix bug in my bugfix (#52337) The fix in #52323 was not quite correct. Fix that and adjust the test to more accurately reflect the actual issue. 29 November 2023, 05:06:41 UTC
de525af extract common code to create a dummy `IRCode` into a utility function (#52329) Simplifies test code. xref: <https://github.com/JuliaLang/julia/pull/52323#pullrequestreview-1751907258>. 28 November 2023, 17:38:41 UTC
68b4587 Remove no-op specializations of `only` [NFC] (#52328) An easily merged subset of @matthias314's #52296, separated from that PR at @mkitti's suggestion. I kept the low-but-nonzero value specializations for Tuple and Named tuple which offer better error messages. They may be removed with a future PR that improves the generic error message. To verify this is indeed a no-op ``` @code_llvm only(Ref(4)) @code_llvm only(Ref(nothing)) @code_llvm only('z') @code_llvm only((4,)) @code_llvm only((nothing,)) only((4,2)) @code_llvm only(Array{Int, 0}(undef)) @code_llvm only((;x=3)) only((;)) only((;x=2, y=4)) ``` Before ``` julia> @code_llvm only(Ref(4)) ; Function Signature: only(Base.RefValue{Int64}) ; @ iterators.jl:1563 within `only` define i64 @julia_only_2451({}* noundef nonnull align 8 dereferenceable(8) %"x::RefValue") #0 { top: ; ┌ @ refvalue.jl:59 within `getindex` ; │┌ @ Base.jl:49 within `getproperty` %0 = bitcast {}* %"x::RefValue" to i64* %.x = load i64, i64* %0, align 8 ; └└ ret i64 %.x } julia> @code_llvm only(Ref(nothing)) ; Function Signature: only(Base.RefValue{Nothing}) ; @ iterators.jl:1563 within `only` define void @julia_only_2519({}* noundef nonnull %"x::RefValue") #0 { top: ret void } julia> @code_llvm only('z') ; Function Signature: only(Char) ; @ iterators.jl:1565 within `only` define i32 @julia_only_2527(i32 zeroext %"x::Char") #0 { top: ret i32 %"x::Char" } julia> @code_llvm only((4,)) ; Function Signature: only(Tuple{Int64}) ; @ iterators.jl:1566 within `only` define i64 @julia_only_2529([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::Tuple") #0 { top: ; ┌ @ tuple.jl:31 within `getindex` %"x::Tuple[1]_ptr" = getelementptr inbounds [1 x i64], [1 x i64]* %"x::Tuple", i64 0, i64 0 ; └ %"x::Tuple[1]_ptr.unbox" = load i64, i64* %"x::Tuple[1]_ptr", align 8 ret i64 %"x::Tuple[1]_ptr.unbox" } julia> @code_llvm only((nothing,)) ; Function Signature: only(Tuple{Nothing}) ; @ iterators.jl:1566 within `only` define void @julia_only_2532() #0 { top: ret void } julia> only((4,2)) ERROR: ArgumentError: Tuple contains 2 elements, must contain exactly 1 element Stacktrace: [1] only(x::Tuple{Int64, Int64}) @ Base.Iterators ./iterators.jl:1567 [2] top-level scope @ REPL[6]:1 julia> @code_llvm only(Array{Int, 0}(undef)) ; Function Signature: only(Array{Int64, 0}) ; @ iterators.jl:1570 within `only` define i64 @julia_only_2779({}* noundef nonnull align 8 dereferenceable(16) %"a::Array") #0 { top: ; ┌ @ abstractarray.jl:1314 within `getindex` ; │┌ @ abstractarray.jl:1343 within `_getindex` ; ││┌ @ essentials.jl:817 within `getindex` %0 = bitcast {}* %"a::Array" to i64** %1 = load i64*, i64** %0, align 8 %2 = load i64, i64* %1, align 8 ; └└└ ret i64 %2 } julia> @code_llvm only((;x=3)) ; Function Signature: only(NamedTuple{(:x,), Tuple{Int64}}) ; @ iterators.jl:1571 within `only` define i64 @julia_only_2794([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::NamedTuple") #0 { top: ; ┌ @ abstractarray.jl:469 within `first` ; │┌ @ namedtuple.jl:165 within `iterate` @ namedtuple.jl:165 %"x::NamedTuple.x_ptr" = getelementptr inbounds [1 x i64], [1 x i64]* %"x::NamedTuple", i64 0, i64 0 ; └└ %"x::NamedTuple.x_ptr.unbox" = load i64, i64* %"x::NamedTuple.x_ptr", align 8 ret i64 %"x::NamedTuple.x_ptr.unbox" } julia> only((;)) ERROR: ArgumentError: NamedTuple contains 0 elements, must contain exactly 1 element Stacktrace: [1] only(x::@NamedTuple{}) @ Base.Iterators ./iterators.jl:1572 [2] top-level scope @ REPL[9]:1 julia> only((;x=2, y=4)) ERROR: ArgumentError: NamedTuple contains 2 elements, must contain exactly 1 element Stacktrace: [1] only(x::@NamedTuple{x::Int64, y::Int64}) @ Base.Iterators ./iterators.jl:1572 [2] top-level scope @ REPL[10]:1 ``` After ``` julia> @code_llvm only(Ref(4)) ; Function Signature: only(Base.RefValue{Int64}) ; @ iterators.jl:1550 within `only` define i64 @julia_only_6821({}* noundef nonnull align 8 dereferenceable(8) %"x::RefValue") #0 { top: ; @ iterators.jl:1551 within `only` ; ┌ @ refpointer.jl:103 within `iterate` ; │┌ @ refvalue.jl:59 within `getindex` ; ││┌ @ Base.jl:49 within `getproperty` %0 = bitcast {}* %"x::RefValue" to i64* %.x = load i64, i64* %0, align 8 ; └└└ ; @ iterators.jl:1559 within `only` ret i64 %.x } julia> @code_llvm only(Ref(nothing)) ; Function Signature: only(Base.RefValue{Nothing}) ; @ iterators.jl:1550 within `only` define void @julia_only_6824({}* noundef nonnull %"x::RefValue") #0 { top: ; @ iterators.jl:1559 within `only` ret void } julia> @code_llvm only('z') ; Function Signature: only(Char) ; @ iterators.jl:1550 within `only` define i32 @julia_only_6826(i32 zeroext %"x::Char") #0 { top: ; @ iterators.jl:1559 within `only` ret i32 %"x::Char" } julia> @code_llvm only((4,)) ; Function Signature: only(Tuple{Int64}) ; @ /Users/x/.julia/dev/julia/base/iterators.jl:1566 within `only` define i64 @julia_only_6833([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::Tuple") #0 { top: ; ┌ @ tuple.jl:31 within `getindex` %"x::Tuple[1]_ptr" = getelementptr inbounds [1 x i64], [1 x i64]* %"x::Tuple", i64 0, i64 0 ; └ %"x::Tuple[1]_ptr.unbox" = load i64, i64* %"x::Tuple[1]_ptr", align 8 ret i64 %"x::Tuple[1]_ptr.unbox" } julia> @code_llvm only((nothing,)) ; Function Signature: only(Tuple{Nothing}) ; @ /Users/x/.julia/dev/julia/base/iterators.jl:1566 within `only` define void @julia_only_6836() #0 { top: ret void } julia> only((4,2)) ERROR: ArgumentError: Tuple contains 2 elements, must contain exactly 1 element Stacktrace: [1] only(x::Tuple{Int64, Int64}) @ Base.Iterators ~/.julia/dev/julia/base/iterators.jl:1564 [2] top-level scope @ REPL[31]:1 julia> @code_llvm only(Array{Int, 0}(undef)) ; Function Signature: only(Array{Int64, 0}) ; @ iterators.jl:1550 within `only` define i64 @julia_only_6848({}* noundef nonnull align 8 dereferenceable(16) %"x::Array") #0 { L60: ; @ iterators.jl:1551 within `only` ; ┌ @ array.jl:884 within `iterate` @ array.jl:884 ; │┌ @ essentials.jl:817 within `getindex` %0 = bitcast {}* %"x::Array" to i64** %1 = load i64*, i64** %0, align 8 %2 = load i64, i64* %1, align 8 ; └└ ; @ iterators.jl:1559 within `only` ret i64 %2 } julia> @code_llvm only((;x=3)) ; Function Signature: only(NamedTuple{(:x,), Tuple{Int64}}) ; @ /Users/x/.julia/dev/julia/base/iterators.jl:1571 within `only` define i64 @julia_only_6871([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::NamedTuple") #0 { top: ; ┌ @ abstractarray.jl:469 within `first` ; │┌ @ namedtuple.jl:165 within `iterate` @ namedtuple.jl:165 %"x::NamedTuple.x_ptr" = getelementptr inbounds [1 x i64], [1 x i64]* %"x::NamedTuple", i64 0, i64 0 ; └└ %"x::NamedTuple.x_ptr.unbox" = load i64, i64* %"x::NamedTuple.x_ptr", align 8 ret i64 %"x::NamedTuple.x_ptr.unbox" } julia> only((;)) ERROR: ArgumentError: NamedTuple contains 0 elements, must contain exactly 1 element Stacktrace: [1] only(x::@NamedTuple{}) @ Base.Iterators ~/.julia/dev/julia/base/iterators.jl:1568 [2] top-level scope @ REPL[34]:1 julia> only((;x=2, y=4)) ERROR: ArgumentError: NamedTuple contains 2 elements, must contain exactly 1 element Stacktrace: [1] only(x::@NamedTuple{x::Int64, y::Int64}) @ Base.Iterators ~/.julia/dev/julia/base/iterators.jl:1568 [2] top-level scope @ REPL[35]:1 ``` Diff ```diff < # Before --- > # After 4,5c4,5 < ; @ iterators.jl:1563 within `only` < define i64 @julia_only_2451({}* noundef nonnull align 8 dereferenceable(8) %"x::RefValue") #0 { --- > ; @ iterators.jl:1550 within `only` > define i64 @julia_only_6821({}* noundef nonnull align 8 dereferenceable(8) %"x::RefValue") #0 { 7,11c7,14 < ; ┌ @ refvalue.jl:59 within `getindex` < ; │┌ @ Base.jl:49 within `getproperty` < %0 = bitcast {}* %"x::RefValue" to i64* < %.x = load i64, i64* %0, align 8 < ; └└ --- > ; @ iterators.jl:1551 within `only` > ; ┌ @ refpointer.jl:103 within `iterate` > ; │┌ @ refvalue.jl:59 within `getindex` > ; ││┌ @ Base.jl:49 within `getproperty` > %0 = bitcast {}* %"x::RefValue" to i64* > %.x = load i64, i64* %0, align 8 > ; └└└ > ; @ iterators.jl:1559 within `only` 17,18c20,21 < ; @ iterators.jl:1563 within `only` < define void @julia_only_2519({}* noundef nonnull %"x::RefValue") #0 { --- > ; @ iterators.jl:1550 within `only` > define void @julia_only_6824({}* noundef nonnull %"x::RefValue") #0 { 19a23 > ; @ iterators.jl:1559 within `only` 25,26c29,30 < ; @ iterators.jl:1565 within `only` < define i32 @julia_only_2527(i32 zeroext %"x::Char") #0 { --- > ; @ iterators.jl:1550 within `only` > define i32 @julia_only_6826(i32 zeroext %"x::Char") #0 { 27a32 > ; @ iterators.jl:1559 within `only` 33,34c38,39 < ; @ iterators.jl:1566 within `only` < define i64 @julia_only_2529([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::Tuple") #0 { --- > ; @ /Users/x/.julia/dev/julia/base/iterators.jl:1566 within `only` > define i64 @julia_only_6833([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::Tuple") #0 { 45,46c50,51 < ; @ iterators.jl:1566 within `only` < define void @julia_only_2532() #0 { --- > ; @ /Users/x/.julia/dev/julia/base/iterators.jl:1566 within `only` > define void @julia_only_6836() #0 { 55c60 < @ Base.Iterators ./iterators.jl:1567 --- > @ Base.Iterators ~/.julia/dev/julia/base/iterators.jl:1564 57c62 < @ REPL[6]:1 --- > @ REPL[31]:1 61,70c66,76 < ; @ iterators.jl:1570 within `only` < define i64 @julia_only_2779({}* noundef nonnull align 8 dereferenceable(16) %"a::Array") #0 { < top: < ; ┌ @ abstractarray.jl:1314 within `getindex` < ; │┌ @ abstractarray.jl:1343 within `_getindex` < ; ││┌ @ essentials.jl:817 within `getindex` < %0 = bitcast {}* %"a::Array" to i64** < %1 = load i64*, i64** %0, align 8 < %2 = load i64, i64* %1, align 8 < ; └└└ --- > ; @ iterators.jl:1550 within `only` > define i64 @julia_only_6848({}* noundef nonnull align 8 dereferenceable(16) %"x::Array") #0 { > L60: > ; @ iterators.jl:1551 within `only` > ; ┌ @ array.jl:884 within `iterate` @ array.jl:884 > ; │┌ @ essentials.jl:817 within `getindex` > %0 = bitcast {}* %"x::Array" to i64** > %1 = load i64*, i64** %0, align 8 > %2 = load i64, i64* %1, align 8 > ; └└ > ; @ iterators.jl:1559 within `only` 76,77c82,83 < ; @ iterators.jl:1571 within `only` < define i64 @julia_only_2794([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::NamedTuple") #0 { --- > ; @ /Users/x/.julia/dev/julia/base/iterators.jl:1571 within `only` > define i64 @julia_only_6871([1 x i64]* nocapture noundef nonnull readonly align 8 dereferenceable(8) %"x::NamedTuple") #0 { 91c97 < @ Base.Iterators ./iterators.jl:1572 --- > @ Base.Iterators ~/.julia/dev/julia/base/iterators.jl:1568 93c99 < @ REPL[9]:1 --- > @ REPL[34]:1 99c105 < @ Base.Iterators ./iterators.jl:1572 --- > @ Base.Iterators ~/.julia/dev/julia/base/iterators.jl:1568 101c107 < @ REPL[10]:1 --- > @ REPL[35]:1 ``` --------- Co-authored-by: matthias314 <matthias314@posteo.net> Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> 28 November 2023, 14:39:40 UTC
df40bab fix invalidations related to `ismutable` (#52170) Related to #52134. It would be nice if the underlying inference issue was fixed but this seems to be a hotfix for now. I have seen this inference problem occurring in Julia v1.9, v1.10, and current `master`. For example, on Julia v1.9.3, I get ```julia julia> code_warntype(ismutable, (Function,)) MethodInstance for ismutable(::Function) from ismutable(x) @ Base reflection.jl:521 Arguments #self#::Core.Const(ismutable) x::Function Body::Any 1 ─ nothing │ nothing │ %3 = Base.typeof(x)::Type{<:Function} │ %4 = Base.getproperty(%3, :name)::Any │ %5 = Base.getproperty(%4, :flags)::Any │ %6 = (%5 & 0x02)::Any │ %7 = (%6 == 0x02)::Any └── return %7 ``` This causes some invalidations when `using OrdinaryDiffEq`. 28 November 2023, 12:19:12 UTC
72dbbdb cfg_simplify: Handle various corner cases with try/catch blocks (#52323) Mostly long-standing bugs related to not handling the virtual predecessor to catch blocks, but also one recent regression from the EnterNode change. 28 November 2023, 05:18:39 UTC
5b2fcb6 Compiler support for optimizing PersistentDict (#51993) This is part of the work to address #51352 by attempting to allow the compiler to perform SRAO on persistent data structures like `PersistentDict` as if they were regular immutable data structures. These sorts of data structures have very complicated internals (with lots of mutation, memory sharing, etc.), but a relatively simple interface. As such, it is unlikely that our compiler will have sufficient power to optimize this interface by analyzing the implementation. We thus need to come up with some other mechanism that gives the compiler license to perform the requisite optimization. One way would be to just hardcode `PersistentDict` into the compiler, optimizing it like any of the other builtin datatypes. However, this is of course very unsatisfying. At the other end of the spectrum would be something like a generic rewrite rule system (e-graphs anyone?) that would let the PersistentDict implementation declare its interface to the compiler and the compiler would use this for optimization (in a perfect world, the actual rewrite would then be checked using some sort of formal methods). I think that would be interesting, but we're very far from even being able to design something like that (at least in Base - experiments with external AbstractInterpreters in this direction are encouraged). This PR tries to come up with a reasonable middle ground, where the compiler gets some knowledge of the protocol hardcoded without having to know about the implementation details of the data structure. The basic ideas is that `Core` provides some magic generic functions that implementations can extend. Semantically, they are not special. They dispatch as usual, and implementations are expected to work properly even in the absence of any compiler optimizations. However, the compiler is semantically permitted to perform structural optimization using these magic generic functions. In the concrete case, this PR introduces the `KeyValue` interface which consists of two generic functions, `get` and `set`. The core optimization is that the compiler is allowed to rewrite any occurrence of `get(set(x, k, v), k)` into `v` without additional legality checks. In particular, the compiler performs no type checks, conversions, etc. The higher level implementation code is expected to do all that. This approach closely matches the general direction we've been taking in external AbstractInterpreters for embedding additional semantics and optimization opportunities into Julia code (although we generally use methods there, rather than full generic functions), so I think we have some evidence that this sort of approach works reasonably well. Nevertheless, this is certainly an experiment and the interface is explicitly declared unstable. ## Current Status This is fully working and implemented, but the optimization currently bails on anything but the simplest cases. Filling all those cases in is not particularly hard, but should be done along with a more invasive refactoring of SROA, so we should figure out the general direction here first and then we can finish all that up in a follow-up cleanup. ## Obligatory benchmark Before: ``` julia> using BenchmarkTools julia> function foo() a = Base.PersistentDict(:a => 1) return a[:a] end foo (generic function with 1 method) julia> @benchmark foo() BenchmarkTools.Trial: 10000 samples with 993 evaluations. Range (min … max): 32.940 ns … 28.754 μs ┊ GC (min … max): 0.00% … 99.76% Time (median): 49.647 ns ┊ GC (median): 0.00% Time (mean ± σ): 57.519 ns ± 333.275 ns ┊ GC (mean ± σ): 10.81% ± 2.22% ▃█▅ ▁▃▅▅▃▁ ▁▃▂ ▂ ▁▂▄▃▅▇███▇▃▁▂▁▁▁▁▁▁▁▁▂▂▅██████▅▂▁▁▁▁▁▁▁▁▁▁▂▃▃▇███▇▆███▆▄▃▃▂▂ ▃ 32.9 ns Histogram: frequency by time 68.6 ns < Memory estimate: 128 bytes, allocs estimate: 4. julia> @code_typed foo() CodeInfo( 1 ─ %1 = invoke Vector{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}}(Base.HashArrayMappedTries.undef::UndefInitializer, 1::Int64)::Vector{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}} │ %2 = %new(Base.HashArrayMappedTries.HAMT{Symbol, Int64}, %1, 0x00000000)::Base.HashArrayMappedTries.HAMT{Symbol, Int64} │ %3 = %new(Base.HashArrayMappedTries.Leaf{Symbol, Int64}, :a, 1)::Base.HashArrayMappedTries.Leaf{Symbol, Int64} │ %4 = Base.getfield(%2, :data)::Vector{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}} │ %5 = $(Expr(:boundscheck, true))::Bool └── goto #5 if not %5 2 ─ %7 = Base.sub_int(1, 1)::Int64 │ %8 = Base.bitcast(UInt64, %7)::UInt64 │ %9 = Base.getfield(%4, :size)::Tuple{Int64} │ %10 = $(Expr(:boundscheck, true))::Bool │ %11 = Base.getfield(%9, 1, %10)::Int64 │ %12 = Base.bitcast(UInt64, %11)::UInt64 │ %13 = Base.ult_int(%8, %12)::Bool └── goto #4 if not %13 3 ─ goto #5 4 ─ %16 = Core.tuple(1)::Tuple{Int64} │ invoke Base.throw_boundserror(%4::Vector{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}}, %16::Tuple{Int64})::Union{} └── unreachable 5 ┄ %19 = Base.getfield(%4, :ref)::MemoryRef{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}} │ %20 = Base.memoryref(%19, 1, false)::MemoryRef{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}} │ Base.memoryrefset!(%20, %3, :not_atomic, false)::MemoryRef{Union{Base.HashArrayMappedTries.HAMT{Symbol, Int64}, Base.HashArrayMappedTries.Leaf{Symbol, Int64}}} └── goto #6 6 ─ %23 = Base.getfield(%2, :bitmap)::UInt32 │ %24 = Base.or_int(%23, 0x00010000)::UInt32 │ Base.setfield!(%2, :bitmap, %24)::UInt32 └── goto #7 7 ─ %27 = %new(Base.PersistentDict{Symbol, Int64}, %2)::Base.PersistentDict{Symbol, Int64} └── goto #8 8 ─ %29 = invoke Base.getindex(%27::Base.PersistentDict{Symbol, Int64}, :a::Symbol)::Int64 └── return %29 ``` After: ``` julia> using BenchmarkTools julia> function foo() a = Base.PersistentDict(:a => 1) return a[:a] end foo (generic function with 1 method) julia> @benchmark foo() BenchmarkTools.Trial: 10000 samples with 1000 evaluations. Range (min … max): 2.459 ns … 11.320 ns ┊ GC (min … max): 0.00% … 0.00% Time (median): 2.460 ns ┊ GC (median): 0.00% Time (mean ± σ): 2.469 ns ± 0.183 ns ┊ GC (mean ± σ): 0.00% ± 0.00% ▂ █ ▁ █ ▂ █▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁█ █ 2.46 ns Histogram: log(frequency) by time 2.47 ns < Memory estimate: 0 bytes, allocs estimate: 0. julia> @code_typed foo() CodeInfo( 1 ─ return 1 ``` 27 November 2023, 17:01:53 UTC
9233a16 Document Base.StatStruct's fields as public (#50177) These fields are documented in the docstring of `stat`, and also mentioned as being public in the style guide, but their types are not documented, nor is `StatStruct` directly documented. --------- Co-authored-by: Jameson Nash <vtjnash@gmail.com> 27 November 2023, 06:01:45 UTC
9dcedaa Memoize `cwstring` when used for env lookup / modification on Windows (#51371) ~This is just me proposing a suggestion from @KristofferC in https://discourse.julialang.org/t/debug-has-massive-performance-impact/103974/22, it's all his code / idea, not mine.~ This PR makes it so that on windows, we pre-allocate an `IdDict` and every time someone looks up environment variables (motivating example here is `@debug` statements), we store the result of `cwstring(::String)` in that `IdDict` so that it doesn't need to be re-computed for future uses. The idea behind this is that people have observed that [using `@debug` is significantly more costly on Windows than other platforms](https://discourse.julialang.org/t/debug-has-massive-performance-impact/103974), even though we have documented in that manual that it should be a really cheap operation. @KristofferC suggests this is due to the fact that [checking environment variables in Windows is more costly](https://discourse.julialang.org/t/debug-has-massive-performance-impact/103974/18). ~The idea here is that we preallocate a `Cwstring` on Windows that just holds the text `"JULIA_DEBUG"`, so that if `access_env(f, "JULIA_DEBUG")` gets called, we don't need to create a new `Cwstring` and then throw it away each time.~ --------- Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com> Co-authored-by: Jameson Nash <vtjnash@gmail.com> 27 November 2023, 06:00:19 UTC
4a18886 More helpful error message for empty `cpu_target` in `Base.julia_cmd` (#52217) Fix #52209. 27 November 2023, 05:27:30 UTC
08d89eb codegen: fix ssa order regression from initializing structs (#52311) 27 November 2023, 04:27:43 UTC
cd4f49a Give a better assertion failure for module serialization issues (#52306) This issue confused me for a few hours. Basically what happened was that I had some IR that verified fine, but did not survive a deserialization round-trip (related to mismatching pointer types, which of course are starting to lose their meaning in LLVM). In this situation, the first assertion hit was the failure to verify the LLVM module at the output, but I had a hard time figuring out where this IR was coming from, since it looked totally corrupted. If LLVM assertions had been enabled, I would have seen the cantFail give a proper error, but since I only had Julia assertions enabled, I did not see this error message. Try to improve this situation by also asserting the absence of serialization errors using Julia assertions, not just LLVM assertions. 26 November 2023, 20:30:58 UTC
c30d45d Make :enter a proper node type (#52300) This is a prepratory commit in anticipation of giving :enter additional responsibilities of entering and restoring dynamic scopes for ScopedValue (c.f. #51352). This commit simply turns `:enter` into its own node type (like the other terminators). The changes are largely mechanical from the `Expr` version, but will make it easier to add additional semantics in a follow up PR. 26 November 2023, 16:05:52 UTC
49c2d89 jltypes: avoid creating lots of duplicates of this function (#52299) Core.Compiler calls this on many functions, creating associated Types inferred for each for some reason. This saves several MB from the system image. 26 November 2023, 05:20:52 UTC
9ea29d9 Make PersistentDict behave like an IdDict (#52193) 25 November 2023, 15:13:27 UTC
ea261ce Automatically :leave the exception frame on the catch edge (#52245) Right now, we require a :leave expression at both the end of a try region and as the first expression in the catch block. This doesn't really make sense. Throwing the exception should leave the exception frame implicitly. This isn't a huge saving, but does save one IR node (and generated call in the native code) per try/catch block. 25 November 2023, 14:16:02 UTC
18c6c66 Bail out of constprop if we already know it's gonna throw an error (#52293) While looking at some other things, I noticed that we call concrete evaluation on every `error("foo")`-like call just to find out again that it errors. which is obviously wasteful. In the future we may want to model exception-`:consistent`-cy in which case there could be some additional `exct` refinement here, but we're not there yet. --------- Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com> 25 November 2023, 14:15:48 UTC
cb01a3b optimized (and ordered) IdSet code (#52114) We have had this smallintset code around for a while for internal purposes, though it was not quite fully general purpose (it didn't have pop). We also have around this tiny global_roots_table (about 1500 entries after building the sysig). This saves about 50% of the space for storing that table. It also optimizes all other IdSet code to not be an inefficient mutable wrapper around an IdDict, but instead be an ordered (by first insertion) set type of its own. 25 November 2023, 04:45:22 UTC
cc4424e Improve type-inference in complex eigen (#52290) Indexing using integers instead of a `Vector` uses constant-propagation to improve the inferred return types. After this, the return type of `eigen(::Matrix{ComplexF64})` is inferred as a small union. ```julia julia> @code_typed eigen(rand(ComplexF64,2,2)) CodeInfo( 1 ─ %1 = invoke LinearAlgebra.:(var"#eigen#94")(true::Bool, true::Bool, LinearAlgebra.eigsortby::typeof(LinearAlgebra.eigsortby), #self#::typeof(eigen), A::Matrix{ComplexF64})::Union{Eigen{ComplexF64, ComplexF64, Matrix{ComplexF64}, Vector{ComplexF64}}, Eigen{ComplexF64, Float64, Matrix{ComplexF64}, Vector{Float64}}} └── return %1 ) => Union{Eigen{ComplexF64, ComplexF64, Matrix{ComplexF64}, Vector{ComplexF64}}, Eigen{ComplexF64, Float64, Matrix{ComplexF64}, Vector{Float64}}} ``` Close #52289 25 November 2023, 02:35:05 UTC
da48487 Add assertion to `_sqrt_quasitriu_diag_block` (#52274) Fixes https://github.com/JuliaLang/julia/issues/52255 Before this change: ```julia julia> using JET julia> report_call(√, Tuple{Matrix{Complex{Float64}}}) ═════ 1 possible error found ═════ ┌ sqrt(A::Matrix{ComplexF64}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/dense.jl:884 │┌ sqrt(A::UpperTriangular{ComplexF64, Matrix{ComplexF64}}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2159 ││┌ sqrt_quasitriu(A0::UpperTriangular{ComplexF64, Matrix{ComplexF64}}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2184 │││┌ sqrt_quasitriu(A0::UpperTriangular{ComplexF64, Matrix{ComplexF64}}; blockwidth::Int64) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2211 ││││┌ kwcall(::@NamedTuple{blockwidth::Int64, n::Int64}, ::typeof(LinearAlgebra._sqrt_quasitriu!), R::Matrix{ComplexF64}, A::UpperTriangular{ComplexF64, Matrix{ComplexF64}}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2226 │││││┌ _sqrt_quasitriu!(R::Matrix{ComplexF64}, A::UpperTriangular{ComplexF64, Matrix{ComplexF64}}; blockwidth::Int64, n::Int64) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2228 ││││││┌ _sqrt_quasitriu_block!(R::Matrix{ComplexF64}, A::UpperTriangular{ComplexF64, Matrix{ComplexF64}}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2248 │││││││┌ _sqrt_quasitriu_diag_block!(R::Matrix{ComplexF64}, A::UpperTriangular{ComplexF64, Matrix{ComplexF64}}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2263 ││││││││┌ _sqrt_real_2x2!(R::SubArray{ComplexF64, 2, Matrix{ComplexF64}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}, A::SubArray{ComplexF64, 2, UpperTriangular{ComplexF64, Matrix{ComplexF64}}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2311 │││││││││┌ _real_sqrt(θ::ComplexF64, μ::Float64) @ LinearAlgebra /home/mason/julia-dev/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/triangular.jl:2323 ││││││││││┌ >=(x::ComplexF64, y::Int64) @ Base ./operators.jl:425 │││││││││││┌ <=(x::Int64, y::ComplexF64) @ Base ./operators.jl:401 ││││││││││││┌ <(x::Int64, y::ComplexF64) @ Base ./operators.jl:352 │││││││││││││ no matching method found `isless(::Int64, ::ComplexF64)`: isless(x::Int64, y::ComplexF64) ││││││││││││└──────────────────── ``` after this change ```julia julia> report_call(√, Tuple{Matrix{Complex{Float64}}}) No errors detected ``` There was a pre-existing comment in the function claiming "this branch is never reached when A is complex triangular". I don't fully understand why that would be the case, but if we believe it to be true, it would be good to share that information with the compiler. I used `@assert` here because it's believed by whoever wrote the algorithm that this branch is unreachable, so I think that's the correct way to use `@assert`, but if people have suggestions for a better way to communicate this info to the compiler, I'd gladly change things around. 25 November 2023, 02:29:21 UTC
3bdf9d5 inference: optimize call to `builtin_tfunction` (#52284) By avoiding unnecessary copy of `argtypes[2:end]`. 25 November 2023, 02:26:01 UTC
c1fc985 irinterp: improve semi-concrete interpretation accuracy (#52275) By enforcing re-inference on calls with all constant arguments. While it's debatable whether this approach is the most efficient, it was the easiest choice given that `used_ssas` based on `IncrementaCompact` wasn't an option for irinterp. - fixes #52202 - fixes #50037 25 November 2023, 02:23:33 UTC
93d4740 Document and export `Base.in!` (#51636) I think `in!` is a useful general function for users, and would be good to have as official API. Its semantics is clear and unambiguous, while providing a clear performance advantage over the naive implementation. For more evidence that this functionality is useful, consider: * Rust's `HashSet::insert` works just like this implementation of `in!` * This function was already used in the implementation of `Base.unique`, precisely for the performance over the naive approach Comes from #45156 with some initial discussion. 25 November 2023, 01:11:40 UTC
9c5def5 docs: simpler example for randn (#52252) As noted here: https://discourse.julialang.org/t/how-do-we-julians-win-big-when-the-situation-is-so-unfair/106433/63?u=lmiq The randn example is overly complicated and does not indicate the simplest usage, which is probably the most typical one. Co-authored-by: Steven G. Johnson <stevenj@mit.edu> 25 November 2023, 01:05:06 UTC
9e8fe68 Turn Method Overwritten Error into a PrecompileError -- turning off caching (#52214) Fixes #52213 Overwritting methods during cache creation is currently not something that the system can support and can lead to surprising, counter-intuitive and fatal errors. In 1.10 we turned it from a warning to a strong error, with this PR it remains a strong error, but the precompilation system recognizes it and essentially sets `__precompile__(false)` for this module and all modules that depend on it. Before: ``` julia> using OverwriteMethodError [ Info: Precompiling OverwriteMethodError [top-level] WARNING: Method definition +(Bool, Bool) in module Base at bool.jl:166 overwritten in module OverwriteMethodError at /home/vchuravy/src/julia2/OverwriteMethodError.jl:2. ERROR: LoadError: Method overwriting is not permitted during Module precompile. Stacktrace: [1] top-level scope @ ~/src/julia2/OverwriteMethodError.jl:2 [2] include @ Base ./Base.jl:489 [inlined] [3] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing) @ Base ./loading.jl:2216 [4] top-level scope @ stdin:3 in expression starting at /home/vchuravy/src/julia2/OverwriteMethodError.jl:1 in expression starting at stdin:3 ERROR: Failed to precompile OverwriteMethodError [top-level] to "/home/vchuravy/.julia/compiled/v1.10/jl_guiuCQ". Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool) @ Base ./loading.jl:2462 [3] compilecache @ Base ./loading.jl:2334 [inlined] [4] (::Base.var"#968#969"{Base.PkgId})() @ Base ./loading.jl:1968 [5] mkpidlock(f::Base.var"#968#969"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool}) @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.0-rc1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:93 [6] #mkpidlock#6 @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.0-rc1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:88 [inlined] [7] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64}) @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.10.0-rc1+0.x64.linux.gnu/share/julia/stdlib/v1.10/FileWatching/src/pidfile.jl:111 [8] #invokelatest#2 @ Base ./essentials.jl:889 [inlined] [9] invokelatest @ Base ./essentials.jl:884 [inlined] [10] maybe_cachefile_lock(f::Base.var"#968#969"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64) @ Base ./loading.jl:2977 [11] maybe_cachefile_lock @ Base ./loading.jl:2974 [inlined] [12] _require(pkg::Base.PkgId, env::String) @ Base ./loading.jl:1964 [13] __require_prelocked(uuidkey::Base.PkgId, env::String) @ Base ./loading.jl:1806 [14] #invoke_in_world#3 @ Base ./essentials.jl:921 [inlined] [15] invoke_in_world @ Base ./essentials.jl:918 [inlined] [16] _require_prelocked(uuidkey::Base.PkgId, env::String) @ Base ./loading.jl:1797 [17] macro expansion @ Base ./loading.jl:1784 [inlined] [18] macro expansion @ Base ./lock.jl:267 [inlined] [19] __require(into::Module, mod::Symbol) @ Base ./loading.jl:1747 [20] #invoke_in_world#3 @ Base ./essentials.jl:921 [inlined] [21] invoke_in_world @ Base ./essentials.jl:918 [inlined] [22] require(into::Module, mod::Symbol) @ Base ./loading.jl:1740 ``` After: ``` julia> using OverwriteMethodError ┌ Info: Precompiling OverwriteMethodError [top-level] └ @ Base loading.jl:2486 WARNING: Method definition +(Bool, Bool) in module Base at bool.jl:166 overwritten in module OverwriteMethodError at /home/vchuravy/src/julia2/OverwriteMethodError.jl:2. ERROR: Method overwriting is not permitted during Module precompile. ┌ Info: Skipping precompilation since __precompile__(false). Importing OverwriteMethodError [top-level]. └ @ Base loading.jl:2084 ``` --------- Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com> 24 November 2023, 22:35:51 UTC
6e23543 Fix off-by-2 in codegen (#52292) This is cherry-picked from #52245. This is an independent bugfix, and looks like #52245 might need another round of discussion. There were two separate off-by-1's in the codegen code that is trying to detect assignments to slots inside try/catch regions. First, it was asking to include the value of the catch label, which is actually the first statement *not* in the try region. Second, there was a confusion of 0 and 1 based indexing in the iteration bounds. The end result of this was that the code was also looking at the first two statements of the catch region. This wasn't a problem before #52245 (other than a potentially over-conservative marking of some slots as volatile), because our catch blocks always had at least two statements (a :leave and a terminator), but with the `:leave` change, it is possible to have catch blocks with only one statement. If these happened to be at the end of the function, things would blow up. As a side node, this code isn't particularly sound, because it assumes that try/catch regions are lexical, which they are not. The assumption happens to work out ok for the code we generate in the frontend and optimized IR doesn't have slots, so we don't use this code, but it is not in general sound. 24 November 2023, 15:26:51 UTC
a386cd1 Fix multiversioning issues caused by the parallel llvm work (#52194) So after struggling with this for a long while it seems there were two different issues. The first one we lacked coverage over, but the other was a very subtle issue when we sorted the fptrs. ~I still need to add test that does multiversioning where we call between multiversioned functions~ Fixes https://github.com/JuliaLang/julia/issues/52079 24 November 2023, 13:51:32 UTC
187e8c2 Add `BracketedSort` a new, faster algorithm for `partialsort` and friends (#52006) 23 November 2023, 21:16:52 UTC
79a845c Don't reuse passmanagers across compilations (#52054) 23 November 2023, 19:04:29 UTC
9884e44 Don't print "No global of this name exists in this module." on UndefValError (#52280) 23 November 2023, 16:16:15 UTC
38438d7 Remove upper bound in timer test (#52282) 23 November 2023, 16:15:22 UTC
0c46852 effects: refine `:nothrow` when `exct` is known to be `Bottom` (#52270) Implements effects refinement that goes in the reverse direction of what was implemented in 8dd0cf5. In the discussion at JuliaLang/julia#52268, there's a debate on how to deal with exceptions like `StackOverflowError` or `InterruptException` that are caused by environment. The original purpose of #52268 was to fix issues where methods that could cause a `StackOverflowError` were mistakenly refined to `:nothrow`. But now it feels like it's a bit weird to model such environment-dependent exceptions as `:nothrow` in the first place. If we can exclude environment-dependent errors from the `:nothrow` modeling, those methods might be safely considered `:nothrow`. To prevent the concrete evaluation of methods that may cause `StackOverflowError`, it's enough to taint its `:teminates` after all. This commit excludes environment-depending errors from the `:nothrow` modeling, reverting the changes from #52268 while making it explicit in the `Effects` and `Base.@assume_effects` docstrings. Anyway, now the compile is able to prove `:nothrow`-ness of the following frame: ```julia function getindex_nothrow(xs::Vector{Int}, i::Int) try return xs[i] catch err err isa BoundsError && return nothing rethrow(err) end end ``` 23 November 2023, 08:00:46 UTC
a624d44 Widen diagonal var during `Type` unwrapping in `instanceof_tfunc` (#52228) close #52168 close #27031 23 November 2023, 07:22:28 UTC
10d58eb reflection: add `Base.infer_return_type` utility (#52247) This commit introduces `Base.infer_return_type`, a new reflection utility which shares a similar interface with `Base.return_types` but differs in its output; `Base.infer_return_type` provides a singular return type taking into account all potential outcomes specified with the given call signature. This function parallels `Base.infer_effects` and the newly added `Base.infer_exception_type`, offering some utility, especially in testing scenarios. 23 November 2023, 04:28:37 UTC
44b8983 compiler: add flag manipulation utilities (#52269) Right now, we're checking if a flag exists using bare operations like `&` and `==`. This works fine, but I think unifying these interfaces could make our code easier to maintain. So, this commit introduces some new high-level operators and refactored the code to incorporate them: - `has_flag(curr::UInt32, flag::UInt32)` - `has_flag(inst::Instruction, flag::UInt32)` - `add_flag!(inst::Instruction, flag::UInt32)` - `sub_flag!(inst::Instruction, flag::UInt32)` 23 November 2023, 04:23:42 UTC
418423b optimize `allunique()` for sorted collections (#50372) 22 November 2023, 22:46:49 UTC
abfc2c6 Add SubArray optimization to sorting (#52072) 22 November 2023, 18:54:09 UTC
05f4b05 fix Unicode.julia_chartransform for Julia 1.10 (#52027) #49559 by @JeffBezanson updated `src/flisp/julia_charmap.h` but missed [the comment](https://github.com/JuliaLang/julia/blob/164969f3d06919b073f3aa9ee608e40974ca82d9/src/flisp/julia_charmap.h#L5-L6) noting that `base/strings/unicode.jl` has to be updated accordingly. 22 November 2023, 16:23:07 UTC
85e3da0 add a compile-time option to enable 4k page sizes (#52229) We're suffering from heavy fragmentation in some of our workloads. Add a build-time option to enable 4k pages (instead of 16k) in the GC, since that improves memory utilization considerably for us. Drawback is that this may increase the number of `madvise` system calls in the sweeping phase by a factor of 4, but concurrent page sweeping should help with some of that. 22 November 2023, 14:26:21 UTC
8c9ac8d Declare `LinearAlgebra.AbstractTriangular` to be public (#52246) Since methods such as `mul!`, `(l/r)mul!` and `(l/r)div!` are defined for `AbstractTriangular`, having this be public allows easier disambiguation in packages. [Several packages](https://juliahub.com/ui/Search?q=AbstractTriangular&type=code) are already using it as if it's public, so it makes sense to support this. This would resolve the specific issue in https://discourse.julialang.org/t/methoderrors-suggest-fixes-that-may-depend-on-internal-bindings/106468 22 November 2023, 13:49:31 UTC
1b40e91 Fix #52216 (#52266) 22 November 2023, 09:01:42 UTC
a0a2bda inference: incorporate stackoverflow possibility into exc type modeling (#52268) Currently `exc_bestguess` does not take into account the possibility of stackoverflow error, causing the inconsistency between exception type modeling and `:nothrow` modeling. This commit fixes it up. 22 November 2023, 09:00:29 UTC
49d3e85 inference: update `exc_bestguess` when seeing unsafe `GotoIfNot` (#52267) 22 November 2023, 08:40:32 UTC
a6c656e Make have_fma consistent between interpreter and compiled (#52206) Currently the interpreter always returns false. Which isn't very good. Make it follow whatever the JIT will do. 21 November 2023, 21:06:36 UTC
0402c78 inference: followups for #51754 (#52241) Composed of: - typeinf_local: factor into `update_cycle_worklists!` utility (78f7b4ecf11b364e543bdfd46aa5fc49b7bdbf42) - inference: fix exception type of `typename` call (fac36d839cb545a1a12991596aafd5584f95bf3b) - add missing type annotations (7ce140ea782c847c2c2253a65301d295b10aa183) - inference: refine exct information if `:nothrow` is proven (76143d37b6563d2da4203747051ef8517453db0d) - ~~improve exception type inference for core math functions (525bd6c2538cbfebc9432df53b0005c9956079be)~~ Separated into another PR 21 November 2023, 11:58:25 UTC
527af66 inference: add reflection utility for exception type analysis (#52240) This commit defines functions that mirror our tools for analyzing return types and computational effects. The key point to discuss is that this commit introduces two functions: `Base.infer_exception_types` and `Base.infer_exception_type`. `Base.infer_exception_types` acts like `Base.return_types`, giving a list of exception types for each method that matches with the given call signature. On the other hand, `Base.infer_exception_type` is akin to `Base.infer_effects`, returning a single exception type that covers all potential outcomes entailed by the given call signature. I personally lean towards the latter for its utility, particularly in testing scenarios, but I included `infer_exception_types` too for consistency with `return_types`. I'd welcome any feedback on this approach. 21 November 2023, 11:55:13 UTC
8dd0cf5 inference: refine `exct` information if `:nothrow` is proven 21 November 2023, 09:53:19 UTC
d85cb0e add missing type annotations 21 November 2023, 09:51:17 UTC
808d7ac inference: fix exception type of `typename` call 21 November 2023, 09:51:17 UTC
da1258c `typeinf_local`: factor into `update_cycle_worklists!` utility 21 November 2023, 09:51:17 UTC
15ab026 inference: add reflection utility for exception type analysis This commit defines functions that mirror our tools for analyzing return types and computational effects. The key point to discuss is that this commit introduces two functions: `Base.exception_types` and `Base.exception_type`. `Base.exception_types` acts like `Base.return_types`, giving a list of exception types for each method that matches with the given call signature. On the other hand, `Base.exception_type` is akin to `Base.infer_effects`, returning a single exception type that covers all potential outcomes entailed by the given call signature. I personally lean towards the latter for its utility, particularly in testing scenarios, but I included `exception_types` too for consistency with `return_types`. I'd welcome any feedback on this approach. 21 November 2023, 09:15:27 UTC
65090d1 tweak the error printed for `PrecompileError` to indicate that it doesn't have to be due to `__precompile__(false)` (#51367) 21 November 2023, 02:08:45 UTC
67161a3 Silence noisy test in `test/cmdlineargs.jl` (#52225) This is spamming ([example](https://buildkite.com/julialang/julia-master/builds/30247#018be235-0cd3-43bb-ba43-cc378ac0d682/792-1056)) ``` From worker 12: ERROR: LoadError: UndefVarError: `@which` not defined in `Main` From worker 12: Stacktrace: From worker 12: [1] top-level scope From worker 12: @ :0 From worker 12: [2] lower From worker 12: @ ./meta.jl:163 [inlined] From worker 12: [3] eval_user_input(errio::IOContext{Base.PipeEndpoint}, ast::Any, show_value::Bool) From worker 12: @ Base ./client.jl:141 From worker 12: [4] run_main_repl(interactive::Bool, quiet::Bool, banner::Symbol, history_file::Bool, color_set::Bool) From worker 12: @ Base ./client.jl:477 From worker 12: [5] repl_main From worker 12: @ Base ./client.jl:561 [inlined] From worker 12: [6] _start() From worker 12: @ Base ./client.jl:535 From worker 12: in expression starting at none:1 ``` in all jobs 20 November 2023, 17:10:15 UTC
5cb0e51 build: ASAN fixes for glibc (#51755) For the `sigsetjmp` bypass; looks like glibc removed the `__libc_siglongjmp` symbol in glibc 2.34, so change to using the approach taking by our `dlopen` wrapper instead. Adopts topolarity's fixes from #50170 Resolves #47698 Co-authored-by: Jameson Nash <vtjnash@gmail.com> 20 November 2023, 15:45:21 UTC
7327a8f effects: add some more test cases for `Base.@propagate_inbounds` (#52236) 20 November 2023, 13:23:39 UTC
72cd63c jitlayers: reduce excess alignment of #52182 (#52210) 20 November 2023, 03:03:15 UTC
c8ca350 inference: Model type propagation through exceptions (#51754) Currently the type of a caught exception is always modeled as `Any`. This isn't a huge problem, because control flow in Julia is generally assumed to be somewhat slow, so the extra type imprecision of not knowing the return type does not matter all that much. However, there are a few situations where it matters. For example: ``` maybe_getindex(A, i) = try; A[i]; catch e; isa(e, BoundsError) && return nothing; rethrow(); end ``` At present, we cannot infer :nothrow for this method, even if that is the only error type that `A[i]` can throw. This is particularly noticable, since we can now optimize away `:nothrow` exception frames entirely (#51674). Note that this PR still does not make the above example particularly efficient (at least interprocedurally), though specialized codegen could be added on top of this to make that happen. It does however improve the inference result. A second major motivation of this change is that reasoning about exception types is likely to be a major aspect of any future work on interface checking (since interfaces imply the absence of MethodErrors), so this PR lays the groundwork for appropriate modeling of these error paths. Note that this PR adds all the required plumbing, but does not yet have a particularly precise model of error types for our builtins, bailing to `Any` for any builtin not known to be `:nothrow`. This can be improved in follow up PRs as required. 19 November 2023, 22:11:17 UTC
f5d189f effects: don't taint `:noub` for `:new` allocations (#52222) After #52169, the UB previously associated with allocations with uninitialized fields has been addressed, so there's no longer a need to taint `:noub` for `:new` allocations during abstract interpretation. I believe, even without #52169, uninitialized field does not inherently leads to UB, but just causes inconsistency of the program, since what actually causes UB is `getfield` that accesses into whatever object, but not the allocation itself. 19 November 2023, 15:50:33 UTC
c07893d Cartesian indexing for SubArrays with non-integer AbstractRanges (#52094) With #50457 now merged, an `AbstractRange` of `CartesianIndex`es should use `CartesianIndexing` in constructing a `SubArray`. I've limited the method to integer ranges, as this is the case where we know for sure that the indexing may be linear. Fixes ```julia julia> view(1:2, StepRangeLen(CartesianIndex(1), CartesianIndex(1), 0)) 0-element view(::UnitRange{Int64}, StepRangeLen(CartesianIndex(1,), CartesianIndex(1,), 0)) with eltype Int64 ``` --------- Co-authored-by: N5N3 <2642243996@qq.com> 19 November 2023, 10:09:19 UTC
ec3911c Add `lmul!` and `rmul!` for `Bidiagonal` (#51777) 19 November 2023, 08:36:43 UTC
19ca07d cfg_simplify: Fix phi node renaming corner case bug (#52224) Depending on the order of basic block, cfg_simplify! could occasionally forget to renumber a phinode edge, causing at best a downstream assertion and at worst a miscompile. The root of this issue is that ordinarily, we detect whether a phinode edge requires later renaming by looking at the last inserted index, and since we process all indices by order, detecting whether we have already inserted a certain value is a simple comparison. However, cfg_simplify! re-arranges the order of basic blocks, so this check no longer works. Instead, we check whether the incoming edge has already been scheduled. This is slightly less precise (because it is possible for the edge itself to not have been scheduled, even though the value has been), but correct and a slight imprecision here at worst causes a negligible compile-time increase. 19 November 2023, 08:29:29 UTC
41a6f26 Thread lattice through escape analysis (#52223) As of #51494 is used in the base pipeline, so external abstract interpreters inherit it by default. To avoid breaking them, make sure we properly pass down the lattice. 19 November 2023, 08:29:15 UTC
e75dd47 [nghttp2_jll] Upgrade to v1.58.0 (#52218) Usual memo to self: * update version in `stdlib/nghttp2_jll/Project.toml` and `deps/nghttp2.version` * update test in `stdlib/nghttp2_jll/test/runtests.jl` * refresh checksums with `make -f contrib/refresh_checksums.mk -j nghttp2` 18 November 2023, 11:32:51 UTC
959b474 docs: update latest stable version (#52215) 18 November 2023, 10:43:17 UTC
e754f20 Add missing type annotation reported by JET (#52207) We already know from https://github.com/lgoettgens/julia/blob/d0a3edddda407302930f3b2a742a7c8ebbcafb5b/base/loading.jl#L1212 that the value is not `nothing`. However, type inference does not know that as the check lives in a different function. 18 November 2023, 06:30:38 UTC
81afdbc codegen: remove UB from uninitialized bitstypes in new (#52169) In the time since the creation of issue #26764, there _is_ now 'a way to say to llvm "I don't care what this value is, but it always has to be the same"' using the `freeze` instruction, so we can use that to instruct LLVM to not give us undefined behavior when users are using uninitialized memory. There should not be an impact if users were already avoiding this paradigm and are fully initializing their structs. Fixes #26764 17 November 2023, 23:44:45 UTC
625bbde build: disable DT_TEXTREL warning for i686 (#52211) This warning was distracting and annoying me, so I looked into whether it was a bug. Turns out the warning is harmless (loading the object just will make a copy of the ~2kb text field into memory). So add the code to disable it, either with different assembly or different linker flags. Default to continuing to use the same assembly. 17 November 2023, 23:43:52 UTC
20440fd Move "Creating Packages" from Pkg.jl docs (#52102) 17 November 2023, 19:46:52 UTC
9aa7980 codegen: ensure i1 bool is widened to i8 before storing (#52189) Teach value_to_pointer to convert primitive types to their stored representation first, to avoid exposing undef bits later (via memcpy). Take this opportunity to also generalizes the support for zext Bool to anywhere inside any struct for changing any bitwidth to a multiple of 8 bytes. This would change a vector like <2 x i4> from occupying i8 to i16 (c.f. LLVM's LangRef), if such an operation were expressible in Julia today. And take this opportunity to do a bit of code cleanup, now that codegen is better and using helpers from LLVM. Fixes #52127 17 November 2023, 18:58:01 UTC
045b6f9 invmod(n::BitInteger): efficient native modular inverses (#52180) Implement algorithm described in https://arxiv.org/pdf/2204.04342.pdf. The algorithm is pleasingly simple and efficient and the generic Julia implementation is also really enjoyable. 17 November 2023, 16:46:13 UTC
234a758 post-opt-analysis: use EA to refine `:effect_free` (#51494) This commit is aiming to integrate EA into the Base compiler pipeline by using it during the post-opt analysis to refine `:effect_free` information. In doing so, this also generalizes `argescapes` field of `Union{InferenceResult,CodeInstance}` to `analysis_results::AnalysisResults` so that it can hold results of multiple post-optimization analyses, where `AnalysisResults` is designed to be linked-list like data structure. This is because an external `AbstractInterpreter`, like `EscapeAnalyzer`, might perform several post-optimization analyses. Honestly speaking, however, I’m not completely satisfied with this solution yet. It might make more sense to require a single post-optimization analysis for every `AbstractInterpreter` always, as like what we do for the other fields like `rettype`. Co-authored-by: Keno Fischer <keno@juliacomputing.com> 17 November 2023, 16:25:51 UTC
1134315 effects: taint :noub for memory operations (#52186) Currently our effects system does not track `:noub` about memory operations. While this sounds unsafe, it's actually not problematic in most contexts. We restrict concrete-evaluation for anything involing memory operations anyway, as mutable values aren't propagated as compile-time constants (except `Symbol`s). However, it turns out that this is actually causing very dangerous situations in REPL completion context, where `REPLInterpreter` enables aggressive concrete-evaluation that propagates mutable values as constants and ignores the `:consisten`-cy requirement that is usually tainted by memory operations. This commit addresses this issue by tainting `:noub` for memory operations. This commit ends up being somewhat extensive because it would cause significant regression in REPL completions if we make memory ops naively taint `:noub`. Complicating this further is the Base's uses of `boundscheck=false` for peak performance of core memory operations, where bounds checking is conducted separately before performing actual memory operations, e.g.: > essentials.jl ```julia function getindex(A::Array, i::Int) @boundscheck ult_int(bitcast(UInt, sub_int(i, 1)), bitcast(UInt, length(A))) || throw_boundserror(A, (i,)) memoryrefget(memoryref(getfield(A, :ref), i, false), :not_atomic, false) end ``` Here `boundscheck=false` should generally taint `:noub` immediately, but in this kind of case `:noub` can actually be `NOUB_IF_NOINBOUNDS` since it has `@boundscheck` that asserts safety of the memory ops. Note that we could employ a similar technique to `getfield` if we change the above implementation to something like: ```julia function getindex(A::Array, i::Int) memoryrefget(memoryref(getfield(A, :ref), i, @_boundscheck), :not_atomic, @_boundscheck) end ``` although this would end up being slower (check the difference at the code LLVM emits on each implementation if interested). To this end, this commit also introduces new `:noub_if_noinbounds` setting to `@assume_effects` and use it within the base implementation. Now `UInt8` is not enough to represent `EffectsOverride` information, the bit is enlarged to `UInt16`. While this approach might not be ideal, I don't think there is a better way to do this. An alternative I tried before making this commit was to introduce new semantics for `Expr(:boundscheck, ...)` where `Expr(:boundscheck, nothing)` will be constant folded to `false` during codegen while during inference time it does not taint `:noub`. I felt this further complicates the already complex complext `:boundscheck` situation, so I decided to go with the approach to expand the settings set of `@assume_effects`. 17 November 2023, 16:21:29 UTC
03bbf91 remove `switch_[to|from]_irinterp` mechanism (#52204) This seems to cause (probably not very profitable) union-splitting all over the place and complicate code generation. Let's just remove it. As a result, this would slightly reduce image sizes too. 17 November 2023, 15:00:13 UTC
8af47bd irinterp: check if `IRCode` has been compacted (#52203) Adds assertion that checks if `IRCode` passed to irinterp or post-opt analysis has been compacted. And using the invariant from the assertion, this commit also propagates `inst::Instruction` for `scan!` callbacks instead of `stmt` and removes `idx::Int` argument since it is redundant with `inst.idx`. 17 November 2023, 14:58:49 UTC
f1f84a2 [OpenBLAS_jll] Upgrade to v0.3.25 (#52178) See the [release notes of v0.3.25](https://github.com/OpenMathLib/OpenBLAS/releases/tag/v0.3.25). Usual memo to self for the upgrade: * update version number in `stdlib/OpenBLAS_jll/Project.toml` * run `make -f contrib/refresh_checksums.mk -j openblas` to update the checksums * update version number and commit hash in `deps/openblas.version` 17 November 2023, 14:23:49 UTC
29be3ef Move heap_size batching code into pair of functions (#51611) Co-authored-by: Gabriel Baraldi <baraldigabriel@gmail.com> 17 November 2023, 13:21:50 UTC
a65bc9a jitlayers: replace sharedbytes intern pool with one that respects alignment (#52182) The llvm optimizations may increase alignment beyond the initial MAX_ALIGN. This pool's alignment was previously only `sizeof(struct { atomic<int> RefCount; size_t Length; char Data[]; })` however, potentially resulting in segfaults at runtime. Fixes #52118. Should make CI much happier. 17 November 2023, 02:55:10 UTC
1cb85ad Revert "Support sorting iterators (#46104)" (#52010) Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com> 17 November 2023, 02:41:28 UTC
f26947b cap the number of GC threads to number of cpu cores (#52192) 17 November 2023, 02:09:09 UTC
b1c8e12 Use pkgimages for coverage & malloc tracking by ignoring native code in tracked packages (#52123) 16 November 2023, 19:51:25 UTC
40e56a5 add some instrumentation to measure page utilization per size class (#52164) One of the limitations is that it's only accurate right after the GC. Still might be helpful for observability purposes. 16 November 2023, 19:15:03 UTC
7f18f76 inference: enhance memoryop tfunc robustness (#52185) Make them able to handle potential `Vararg` argument. Also adds plenty of tests. 16 November 2023, 16:51:06 UTC
4689850 Compatible parent index when displaying dots in SubArrays (#52097) Fix #52095 by passing the `reindex`ed indices through `to_indices` to convert them to `Integer`s. This should strip `CartesianIndex`es in particular, which resolves the issue. Fixes: ```julia julia> view(1:2, [CartesianIndex(2)]) 1-element view(::UnitRange{Int64}, CartesianIndex{1}[CartesianIndex(2,)]) with eltype Int64: 2 ``` 16 November 2023, 16:14:45 UTC
2fb06a7 inference: streamline `builtin_effects` (#52177) Composed of the following two commits: --- > stop passing `arginfo` to `builtin_effects` (e758d079d7f1aa185f2e90273be0340d669dfa3e) `builtin_effects` no longer requires the `arginfo` parameter, as `getfield_effects` doesn't depend on analyzing `fargs` for `:boundscheck` information anymore. The equivalent analysis is now done as the post-opt analysis. Given that `:boundscheck` expressions are now lowered to an independent statement, even if we need to analyze `fargs` within `builtin_effects`, we would likely need to pass `sv::AbsIntState` too in order to allow it to use `ssa_def_slot`. For now, it's more efficient to just eliminate the passing of `arginfo` since it is not necessary. --- > exclude the first builtin function from `argtypes` of `builtin_effects` (90c7477223f72839a2927a1ef6b3d293fde6a945) Streamlined the `builtin_effects` function to accept `argtypes` without including the builtin function itself. The first argument in the previous implementation of `argtypes` was just redundant. This change might reduce unnecessary allocations. --- 16 November 2023, 07:19:22 UTC
574c5d1 exclude the first builtin function from `argtypes` of `builtin_effects` Streamlined the `builtin_effects` function to accept `argtypes` without including the builtin function itself. The first argument in the previous implementation of `argtypes` was just redundant. This change might reduce unnecessary allocations. 16 November 2023, 05:29:13 UTC
3c82403 stop passing `arginfo` to `builtin_effects` `builtin_effects` no longer requires the `arginfo` parameter, as `getfield_effects` doesn't depend on analyzing `fargs` for `:boundscheck` information anymore. The equivalent analysis is now done as the post-opt analysis. Given that `:boundscheck` expressions are now lowered to an independent statement, even if we need to analyze `fargs` within `builtin_effects`, we would likely need to pass `sv::AbsIntState` too in order to allow it to use `ssa_def_slot`. For now, it's more efficient to just eliminate the passing of `arginfo` since it is not necessary. 16 November 2023, 05:29:13 UTC
253cddc enhance maintainability of `jl_ir_` accessors (#52167) By allowing us to change data size at one place when we make a change to the layout of compressed IR. 16 November 2023, 05:27:32 UTC
aa00204 🤖 [master] Bump the SparseArrays stdlib from 7786a73 to f154de2 (#52183) Co-authored-by: Dilum Aluthge <dilum@aluthge.com> 16 November 2023, 00:54:28 UTC
221f074 🤖 [master] Bump the SparseArrays stdlib from 37fc321 to 7786a73 (#52181) Stdlib: SparseArrays URL: https://github.com/JuliaSparse/SparseArrays.jl.git Stdlib branch: main Julia branch: master Old commit: 37fc321 New commit: 7786a73 Julia version: 1.11.0-DEV SparseArrays version: 1.11.0 Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: https://github.com/JuliaSparse/SparseArrays.jl/compare/37fc321e28a32f79a928c24e5739a83d92de5205...7786a73cefe4147fb423a5fa529aa583ca6c877c ``` $ git log --oneline 37fc321..7786a73 7786a73 Add Aqua compat. Create CompatHelper.yml (#470) ``` Co-authored-by: Dilum Aluthge <dilum@aluthge.com> 15 November 2023, 20:08:06 UTC
5edcdc5 [REPL] fix computation of startpos for path completions (#52009) Fixes https://github.com/JuliaLang/julia/issues/51985 Ensure that the REPL completions escape and unescape text correctly, using the correct functions, and accounting for exactly what the user has currently typed. The old broken method is left around for Pkg, since it has an over-reliance on it returning incorrect answers. Once Pkg is fixed, we can delete that code. Co-authored-by: Jameson Nash <vtjnash@gmail.com> 15 November 2023, 19:15:16 UTC
eaef647 Make c func `abspath` consistent on Windows. Fix tracking path conversion. (#52140) Explanation for the `GetFullPathName` behavior https://developercommunity.visualstudio.com/t/GetFullPath-fails-if-given-empty-string/483359#T-N486167 15 November 2023, 02:42:19 UTC
25bcc60 tidy up the inlining algorithm (#52158) 15 November 2023, 01:38:15 UTC
539ca89 atexit: move hook before precompile output (#51849) To show how this is used, this updates the profile_printing_listener background job to use this mechanism. 15 November 2023, 01:34:47 UTC
a26e23a make objectid effects total (#52119) Avoids the need to rehash most dictionaries on reload. System image data size increase is about 109MB -> 112MB, since there are about 130k Arrays, 75k CodeInstances, 30k Methods, 80k TypeMapEntries, 75k MethodInstance, and 36k Core.Bindings, and other mutable objects. 15 November 2023, 01:13:40 UTC
back to top