Revision history - refs/heads/avi/46713 - snapshot: a72e953ecd624a7df6e6196bbdd05851996c5e40

swh:1:snp:a72e953ecd624a7df6e6196bbdd05851996c5e40

Newer
Older

Revision	Author	Date	Message	Commit Date
d70b8cc	Shuhei Kadowaki	14 September 2022, 10:25:08 UTC	compiler: simplify backedge calculation	16 September 2022, 04:25:04 UTC
4adb03a	Shuhei Kadowaki	13 September 2022, 09:42:21 UTC	compiler: make sure to inline hot lattice ops	16 September 2022, 04:25:04 UTC
0fabe54	Thomas Christensen	15 September 2022, 15:29:38 UTC	Help inference of uninferred callers of `read(::IO, ::Type{String})` (#46577)	15 September 2022, 15:29:38 UTC
4026246	Keno Fischer	15 September 2022, 14:46:08 UTC	Add effect modeling for Core.compilerbarrier (#46772) The compiler barrier itself doesn't have any effects, though of course by pessimizing type information, it may prevent the compiler from discovering effect information it would have otherwise obtained. Nevertheless, particularly the weaker barriers like `:const` and `:conditional` should not, by themselves taint any effects.	15 September 2022, 14:46:08 UTC
00ea165	Keno Fischer	15 September 2022, 14:45:47 UTC	Cache binding pointer in GlobalRef (#46729) On certain workloads, profiling shows a surprisingly high fraction of inference time spent looking up bindings in modules. Bindings use a hash table, so they're expected to be quite fast, but certainly not zero. A big contributor to the problem is that we do basically treat it as zero, looking up bindings for GlobalRefs multiple times for each statement (e.g. in `isconst`, `isdefined`, to get the types, etc). This PR attempts to improve the situation by adding an extra field to GlobalRef that caches this lookup. This field is not serialized and if not set, we fallback to the previous behavior. I would expect the memory overhead to be relatively small, since we do intern GlobalRefs in memory to only have one per binding (rather than one per use). # Benchmarks The benchmarks look quite promising. Consider this artifical example (though it's actually not all that far fetched, given some of the generated code we see): ``` using Core.Intrinsics: add_int const ONE = 1 @eval function f(x, y) z = 0 $([:(z = add_int(x, add_int(z, ONE))) for _ = 1:10000]...) return add_int(z, y) end g(y) = f(ONE, y) ``` On master: ``` julia> @time @code_typed g(1) 1.427227 seconds (1.31 M allocations: 58.809 MiB, 5.73% gc time, 99.96% compilation time) CodeInfo( 1 ─ %1 = invoke Main.f(Main.ONE::Int64, y::Int64)::Int64 └── return %1 ) => Int64 ``` On this PR: ``` julia> @time @code_typed g(1) 0.503151 seconds (1.19 M allocations: 53.641 MiB, 5.10% gc time, 33.59% compilation time) CodeInfo( 1 ─ %1 = invoke Main.f(Main.ONE::Int64, y::Int64)::Int64 └── return %1 ) => Int64 ``` I don't expect the same speedup on other workloads, but there should be a few % speedup on most workloads still. # Future extensions The other motivation for this is that with a view towards #40399, we will want to more clearly define when bindings get resolved. The idea here would then be that binding resolution replaces generic `GlobalRefs` by GlobalRefs with a set binding cache, and any unresolved bindings would be treated conservatively by inference and optimization.	15 September 2022, 14:45:47 UTC
6bbc006	pchintalapudi	15 September 2022, 14:36:24 UTC	Add LLVM pipeline tests (#46542) * Add LLVM pipeline tests * Create per-pipeline optimization tests * Actually run the tests, drop unnecessary output * Move to lit tests instead of julia tests * Apply suggestions from code review `=` -> `in` Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com> Co-authored-by: Valentin Churavy <vchuravy@users.noreply.github.com>	15 September 2022, 14:36:24 UTC
5e3e58e	Kristoffer Carlsson	15 September 2022, 14:13:17 UTC	Revert "Add ability to add output prefixes to the REPL output and use that to implement an IPython mode (#46474)" (#46780) This reverts commit e1188455456035ff6a80ed20e424aa5c8ebf437d.	15 September 2022, 14:13:17 UTC
e118845	Kristoffer Carlsson	15 September 2022, 12:24:25 UTC	Add ability to add output prefixes to the REPL output and use that to implement an IPython mode (#46474)	15 September 2022, 12:24:25 UTC
6557542	N5N3	15 September 2022, 11:43:57 UTC	Also merge var with unchanged bounds. (#46757) * Also merge var with unchanged bounds. The current `env` might not be valid if the var's bounds get fixed by another Unions decision. * Replace `v->var->lb` with `simple_meet` * Remove the unneeded NUll check. Co-Authored-By: Jameson Nash <vtjnash+github@gmail.com>	15 September 2022, 11:43:57 UTC
aae8c48	Daniel Karrasch	15 September 2022, 10:30:02 UTC	Specialize `inv` for cholesky of `Diagonal` (#46763)	15 September 2022, 10:30:02 UTC
997e336	Shuhei Kadowaki	15 September 2022, 06:56:19 UTC	inference: fixes and improvements for backedge computation (#46741) This commit consists of the following changes: * inference: setup separate functions for each backedge kind Also changes the argument list so that they are ordered as `(caller, [backedge information])`. * inference: fix backedge computation for const-prop'ed callsite With this commit `abstract_call_method_with_const_args` doesn't add backedge but rather returns the backedge to the caller, letting the callers like `abstract_call_gf_by_type` and `abstract_invoke` take the responsibility to add backedge to current context appropriately. As a result, this fixes the backedge calculation for const-prop'ed `invoke` callsite. For example, for the following call graph, ```julia foo(a::Int) = a > 0 ? :int : println(a) foo(a::Integer) = a > 0 ? "integer" : println(a) bar(a::Int) = @invoke foo(a::Integer) ``` Previously we added the wrong backedge `nothing, bar(Int64) from bar(Int64)`: ```julia julia> last(only(code_typed(()->bar(42)))) String julia> let m = only(methods(foo, (UInt,))) @eval Core.Compiler for (sig, caller) in BackedgeIterator($m.specializations[1].backedges) println(sig, ", ", caller) end end Tuple{typeof(Main.foo), Integer}, bar(Int64) from bar(Int64) nothing, bar(Int64) from bar(Int64) ``` but now we only add `invoke`-backedge: ```julia julia> last(only(code_typed(()->bar(42)))) String julia> let m = only(methods(foo, (UInt,))) @eval Core.Compiler for (sig, caller) in BackedgeIterator($m.specializations[1].backedges) println(sig, ", ", caller) end end Tuple{typeof(Main.foo), Integer}, bar(Int64) from bar(Int64) ``` * inference: make `BackedgePair` struct * add invalidation test for `invoke` call * optimizer: fixup inlining backedge calculation Should fix the following backedge calculation: ```julia julia> m = which(unique, Tuple{Any}) unique(itr) @ Base set.jl:170 julia> specs = collect(Iterators.filter(m.specializations) do mi mi === nothing && return false return mi.specTypes.parameters[end] === Vector{Int} # find specialization of `unique(::Any)` for `::Vector{Int}` end) Any[] julia> Base._unique_dims([1,2,3],:) # no existing callers with specialization `Vector{Int}`, let's make one 3-element Vector{Int64}: 1 2 3 julia> mi = only(Iterators.filter(m.specializations) do mi mi === nothing && return false return mi.specTypes.parameters[end] === Vector{Int} # find specialization of `unique(::Any)` for `::Vector{Int}` end) MethodInstance for unique(::Vector{Int64}) julia> mi.def unique(itr) @ Base set.jl:170 julia> mi.backedges 3-element Vector{Any}: Tuple{typeof(unique), Any} MethodInstance for Base._unique_dims(::Vector{Int64}, ::Colon) MethodInstance for Base._unique_dims(::Vector{Int64}, ::Colon) # <= now we don't register this backedge ```	15 September 2022, 06:56:19 UTC
94c3a15	Keno Fischer	15 September 2022, 05:24:11 UTC	Improve nothrow analysis of :new with missing sparam (#46754) Similar to #46693, but for :new, rather than getfield. Unfortunately, this is somewhat limited as we don't really have the ability to encode type equality constraints in the lattice. In particular, it would be nice to analyze: ``` struct Foo{T} x::T Foo(x::T) where {T} = new{T}(x) end ``` and be able to prove this nothrow. If it's really important, we could probably pattern match it, but for the moment, this is not easy to do. Nevertheless, we can do something about the similar, but simpler pattern ``` struct Foo{T} x Foo(x::T) where {T} = new{T}(x) end ``` which is what this PR does.	15 September 2022, 05:24:11 UTC
e758982	Hendrik Ranocha	15 September 2022, 03:18:52 UTC	improve type stability of `tail/front(::NamedTuple)` (#46762) This fixes some invalidations when loading ChainRulesCore.jl.	15 September 2022, 03:18:52 UTC
9b1780d	Hendrik Ranocha	15 September 2022, 03:17:33 UTC	fix invalidations from loading Static.jl (#46761) * improve type stability of `sort!(v::AbstractVector, lo::Integer, hi::Integer, ::InsertionSortAlg, o::Ordering)` * improve type stability of `fmt(buf, pos, arg, spec::Spec{T}) where {T <: Strings}` This fixes some invalidations when loading Static.jl.	15 September 2022, 03:17:33 UTC
08115d2	Steven G. Johnson	15 September 2022, 03:16:20 UTC	clarify show docs (#46765)	15 September 2022, 03:16:20 UTC
54c06ad	Alexis	15 September 2022, 03:14:39 UTC	Add reflect! and rotate! in the documentation (#46769)	15 September 2022, 03:14:39 UTC
559e04c	Mark Kittisopikul	15 September 2022, 03:12:25 UTC	Set USE_BINARYBUILDER_OBJCONV=0 when USE_BINARYBUILDER_OPENBLAS=0 (#46726) Co-authored-by: Gabriel Baraldi <baraldigabriel@gmail.com>	15 September 2022, 03:12:25 UTC
f5842f8	Hendrik Ranocha	15 September 2022, 01:27:14 UTC	bump the Tar stdlib from 5606269 to 5914ef9 (#46731)	15 September 2022, 01:27:14 UTC
bd6d170	woclass	14 September 2022, 22:32:48 UTC	Build/win: Build with MSYS2 (#46140) * Makefile: MSYS2: close path conversion for `DEP_LIBS` Automatic path conversion will replace `:` with `;` * Makefile: MSYS2: use `cygpath` for path convert ref: https://www.msys2.org/docs/filesystem-paths/#manual-unix-windows-path-conversion * devdoc/win/msys2: add build steps * devdoc/win/msys2: Add x86/x64 build notes * devdoc/win/msys2: apply sugestions Co-Authored-By: Elliot Saba <staticfloat@gmail.com> * Instead of `CC_WITH_ENV`, scope environment variables to targets * Fix whitespace Co-authored-by: Elliot Saba <staticfloat@gmail.com>	14 September 2022, 22:32:48 UTC
c6abccf	Peter C. Jentsch	14 September 2022, 19:40:11 UTC	Fix symdiff/symdiff! for vector inputs with duplicates, closes #34686 (#46397)	14 September 2022, 19:40:11 UTC
2cfcc1a	pchintalapudi	14 September 2022, 18:04:37 UTC	gitignore lit test times (#46766)	14 September 2022, 18:04:37 UTC
b4af0e5	Hendrik Ranocha	14 September 2022, 13:19:42 UTC	improve type stability of `process_overrides(artifact_dict::Dict, pkk_uuid::Base.UUID)` (#46661) * improve type stability of `process_overrides(artifact_dict::Dict, pkg_uuid::Base.UUID)` This fixes some invalidations when loading Static.jl	14 September 2022, 13:19:42 UTC
35191ad	Jishnu Bhattacharya	14 September 2022, 08:37:40 UTC	Improve argmin/argmax docstring (#46737) The current phrasing "Return a value `x` in the domain of `f`" is a bit misleading, as the domain of `f` may be much larger than the argument `domain`. It's better to be clear that the value returned will be an element of the `domain` that is provided.	14 September 2022, 08:37:40 UTC
b1f8d16	Kristoffer Carlsson	14 September 2022, 07:44:25 UTC	only store version of packages during load time when generating sysimage (#46738)	14 September 2022, 07:44:25 UTC
100d7e8	Udoh Jeremiah	14 September 2022, 07:33:07 UTC	doc: clarify docstring for oftype (#46744)	14 September 2022, 07:33:07 UTC
b368e26	Udoh Jeremiah	14 September 2022, 07:28:59 UTC	typo error (#46752)	14 September 2022, 07:28:59 UTC
0de477a	Jeff Bezanson	14 September 2022, 06:36:26 UTC	change logic for enabling GDB JIT event listener (#46747) - default to false in release build, true in debug build - ENABLE_GDBLISTENER env var overrides that in all builds	14 September 2022, 06:36:26 UTC
8455c8f	Shuhei Kadowaki	14 September 2022, 05:17:51 UTC	inference: concretize `invoke` callsite correctly (#46743) It turns out that previously we didn't concretize `invoke` callsite correctly, as we didn't take into account whether the call being concretized is `invoke`-d or not, e.g.: ``` julia> invoke_concretized2(a::Int) = a > 0 ? :int : nothing invoke_concretized2 (generic function with 1 method) julia> invoke_concretized2(a::Integer) = a > 0 ? :integer : nothing invoke_concretized2 (generic function with 2 methods) julia> let Base.Experimental.@force_compile Base.@invoke invoke_concretized2(42::Integer) end :int # this should return `:integer` instead ``` This commit fixes that up by propagating information `invoke`-d callsite to `concrete_eval_call`. Now we should be able to pass the following test cases: ```julia invoke_concretized1(a::Int) = a > 0 ? :int : nothing invoke_concretized1(a::Integer) = a > 0 ? "integer" : nothing @test Base.infer_effects((Int,)) do a @invoke invoke_concretized1(a::Integer) end \|> Core.Compiler.is_foldable @test Base.return_types() do @invoke invoke_concretized1(42::Integer) end \|> only === String invoke_concretized2(a::Int) = a > 0 ? :int : nothing invoke_concretized2(a::Integer) = a > 0 ? :integer : nothing @test Base.infer_effects((Int,)) do a @invoke invoke_concretized2(a::Integer) end \|> Core.Compiler.is_foldable @test let Base.Experimental.@force_compile @invoke invoke_concretized2(42::Integer) end === :integer ```	14 September 2022, 05:17:51 UTC
7cbea73	Shuhei Kadowaki	14 September 2022, 04:57:44 UTC	Test: fixup issues detected by JET (#46730) - removes stale definition of `test_approx_eq` (this seems to have been reserved for the 0.7 compat, but is currently broken as `full` is really not defined anymore) - fixes invalid construction of `Fail` object - makes `LogTestFailure` more type stable struct - improves type stabilities slightly	14 September 2022, 04:57:44 UTC
f7dea04	Viral B. Shah	14 September 2022, 04:39:00 UTC	Fix whitespace introduced in #45854 (#46750) * Fix whitespace introduced in #45854 * Fix more whitespace Co-authored-by: Keno Fischer <keno@juliacomputing.com>	14 September 2022, 04:39:00 UTC
4bd59ac	TheCedarPrince	14 September 2022, 01:46:05 UTC	Example Workflow Docs on How to Create Tests for One's Own Package (#46357) Co-authored-by: Viral B. Shah <ViralBShah@users.noreply.github.com> Co-authored-by: woclass <git@wo-class.cn>	14 September 2022, 01:46:05 UTC
c5098a9	woclass	14 September 2022, 01:43:37 UTC	[devdoc] Add section “Update the version number of a dependency” (#45854) * [devdoc] Add section “Update the version number of a dependency” * [devdoc] add a check list Co-authored-by: Viral B. Shah <ViralBShah@users.noreply.github.com>	14 September 2022, 01:43:37 UTC
b97a629	Martín D. Maas	14 September 2022, 01:42:13 UTC	doc: highlight recommendation of eachindex in arrays.md (#45341) * Update arrays.md Highlighted two portions of the text as a note and a warning, in view of the discussion Co-authored-by: Mosè Giordano <giordano@users.noreply.github.com> Co-authored-by: Viral B. Shah <ViralBShah@users.noreply.github.com> Co-authored-by: Johnny Chen <johnnychen94@hotmail.com> Co-authored-by: Michael Abbott <32575566+mcabbott@users.noreply.github.com>	14 September 2022, 01:42:13 UTC
6643090	Udoh Jeremiah	14 September 2022, 01:20:43 UTC	Update doc on global variables type annotations (#46686) Co-authored-by: Kristoffer Carlsson <kcarlsson89@gmail.com>	14 September 2022, 01:20:43 UTC
f9b362e	Keno Fischer	13 September 2022, 22:48:47 UTC	sparam inlining: Handle undefined sparams (#46703) `_compute_sparams` is not required to be able to resolve an sparam, it can leave it as a `TypeVar`, in which case accessing it should throw an UndefVarError. This needs to be appropriately handled in inlining. We have a test case like this in the test suite, but it was being papered over by an incorrect check in can_inline_typevar. Remove that check and implement proper undef checking in inlining.	13 September 2022, 22:48:47 UTC
1916d35	Jeff Bezanson	13 September 2022, 20:40:40 UTC	add `nospecialize` to deepcopy entry point (#46692)	13 September 2022, 20:40:40 UTC
d97dd8e	Jameson Nash	13 September 2022, 18:55:50 UTC	handle empty Tuple{} in _methods_by_ftype (#46685) This is silly, but some packages try very hard to precompile everything including this, even though this is invalid. Fix #46295	13 September 2022, 18:55:50 UTC
70bfa3f	Hendrik Ranocha	13 September 2022, 12:08:51 UTC	improve type stability of `lt(p::Perm, a::Integer, b::Integer)` (#46732) This fixes a few hundred invalidations when loading Static/jl/ArrayInterface.jl.	13 September 2022, 12:08:51 UTC
df955b8	Shuhei Kadowaki	13 September 2022, 12:01:26 UTC	compiler: add missing `@nospecialize` annotations (#46734)	13 September 2022, 12:01:26 UTC
85fac87	Erik Schnetter	13 September 2022, 08:05:11 UTC	LinearAlgebra: Allow arrays to be zero-preserving (#46340)	13 September 2022, 08:05:11 UTC