38256f1 | Christopher Rackauckas | 07 February 2020, 22:55:02 UTC | Update Project.toml | 07 February 2020, 22:55:02 UTC |
ece159d | Christopher Rackauckas | 07 February 2020, 22:54:23 UTC | Merge pull request #141 from JuliaDiffEq/fast_adjoints fast adjoints for fast layers | 07 February 2020, 22:54:23 UTC |
8167068 | Chris Rackauckas | 07 February 2020, 19:52:55 UTC | fix typo | 07 February 2020, 19:52:55 UTC |
fcc7454 | Chris Rackauckas | 07 February 2020, 19:32:15 UTC | specialize tanh derivative | 07 February 2020, 19:32:15 UTC |
e21b9d1 | Chris Rackauckas | 07 February 2020, 17:32:29 UTC | fully remove unsafe arrays | 07 February 2020, 17:32:29 UTC |
66fc811 | Yingbo Ma | 07 February 2020, 17:23:54 UTC | Remove UnsafeArrays | 07 February 2020, 17:23:54 UTC |
e723541 | Chris Rackauckas | 07 February 2020, 14:08:08 UTC | mark fixed test | 07 February 2020, 14:08:08 UTC |
86e8bc3 | Chris Rackauckas | 07 February 2020, 13:22:51 UTC | fix test with using | 07 February 2020, 13:22:51 UTC |
19753d9 | Chris Rackauckas | 07 February 2020, 05:30:17 UTC | Make nonlinear optimal control use fast layers | 07 February 2020, 05:30:17 UTC |
25c97af | Chris Rackauckas | 07 February 2020, 05:13:24 UTC | make README example a little nicer | 07 February 2020, 05:13:24 UTC |
f903360 | Chris Rackauckas | 07 February 2020, 04:59:50 UTC | fast adjoints for fast layers Pulls about another 2x acceleration out. ```julia using DiffEqFlux fd = FastDense(2,25,tanh) pd = initial_params(fd) fs = StaticDense(2,25,tanh) x = rand(2) using BenchmarkTools @btime Flux.Zygote.gradient((x,p)->sum(fd(x,p)),x,pd) @btime Flux.Zygote.gradient((x,p)->sum(fs(x,p)),x,pd) # FastDense before: 7.900 μs (100 allocations: 7.69 KiB) 8.066 μs (100 allocations: 7.69 KiB) # FastDense after: 5.283 μs (38 allocations: 3.98 KiB) 5.267 μs (38 allocations: 3.98 KiB) # StaticDense 5.000 μs (25 allocations: 2.73 KiB) 5.067 μs (25 allocations: 2.73 KiB) ``` | 07 February 2020, 04:59:50 UTC |
9ea5d67 | Christopher Rackauckas | 06 February 2020, 21:57:21 UTC | Merge pull request #137 from JuliaDiffEq/gpu fix a bunch of GPU tests | 06 February 2020, 21:57:21 UTC |
9c366c8 | Chris Rackauckas | 06 February 2020, 21:23:36 UTC | comment broken tests | 06 February 2020, 21:23:36 UTC |
0d4fab6 | Chris Rackauckas | 05 February 2020, 15:45:02 UTC | fix a bunch of GPU tests | 06 February 2020, 21:23:06 UTC |
364ea75 | Christopher Rackauckas | 06 February 2020, 20:46:20 UTC | Update Project.toml | 06 February 2020, 20:46:20 UTC |
8fd8a4f | Chris Rackauckas | 06 February 2020, 19:09:09 UTC | fix SDE for sciml_train | 06 February 2020, 19:09:09 UTC |
448cc00 | Christopher Rackauckas | 06 February 2020, 18:36:09 UTC | Merge pull request #139 from ric-cioffi/patch-1 stopping criterion on current loss | 06 February 2020, 18:36:09 UTC |
498aa89 | Chris Rackauckas | 06 February 2020, 15:55:12 UTC | fix Optim setup | 06 February 2020, 15:55:12 UTC |
52f13be | Chris Rackauckas | 06 February 2020, 14:09:19 UTC | sciml_train! -> sciml_train and boolean for halting in a safe way | 06 February 2020, 14:09:19 UTC |
d0017f9 | Christopher Rackauckas | 05 February 2020, 23:44:31 UTC | Update appveyor.yml | 05 February 2020, 23:44:31 UTC |
0d51a5e | ric-cioffi | 05 February 2020, 22:37:30 UTC | stopping criterion on current loss We can make this directly depend on the callback function (which is consistent with Optim). For example a reasonable callback function would be function cb(θ, l) println("Current loss: ", l) return l < 1e-4 end | 05 February 2020, 22:37:30 UTC |
6331437 | Christopher Rackauckas | 04 February 2020, 20:34:25 UTC | Update Project.toml | 04 February 2020, 20:34:25 UTC |
fb58ee4 | Christopher Rackauckas | 04 February 2020, 20:34:09 UTC | Merge pull request #136 from JuliaDiffEq/abstract add some abstract types | 04 February 2020, 20:34:09 UTC |
b261c3a | Christopher Rackauckas | 04 February 2020, 19:53:24 UTC | Update fast_layers.jl | 04 February 2020, 19:53:24 UTC |
0a0d436 | Chris Rackauckas | 04 February 2020, 19:09:23 UTC | add some abstract types | 04 February 2020, 19:09:23 UTC |
49a0f9d | Chris Rackauckas | 02 February 2020, 22:37:39 UTC | patch version | 02 February 2020, 22:37:39 UTC |
2623712 | Chris Rackauckas | 02 February 2020, 22:37:25 UTC | Merge remote-tracking branch 'origin/master' | 02 February 2020, 22:37:25 UTC |
9e650d3 | Chris Rackauckas | 02 February 2020, 22:37:20 UTC | add missing paramlength | 02 February 2020, 22:37:20 UTC |
bb0a9f4 | Christopher Rackauckas | 02 February 2020, 20:22:30 UTC | Update Project.toml | 02 February 2020, 20:22:30 UTC |
9503182 | Christopher Rackauckas | 02 February 2020, 19:11:33 UTC | Update Project.toml | 02 February 2020, 19:11:33 UTC |
ffa8de3 | Christopher Rackauckas | 02 February 2020, 19:11:12 UTC | Merge pull request #131 from JuliaDiffEq/compathelper/new_version/2020-02-02-19-06-06-079-2925281732 CompatHelper: add new compat entry for "UnsafeArrays" at version "1.0" | 02 February 2020, 19:11:12 UTC |
540535e | Christopher Rackauckas | 02 February 2020, 19:10:59 UTC | Merge pull request #130 from JuliaDiffEq/compathelper/new_version/2020-02-02-19-06-01-939-805572191 CompatHelper: add new compat entry for "StaticArrays" at version "0.12" | 02 February 2020, 19:10:59 UTC |
45b6c14 | Christopher Rackauckas | 02 February 2020, 19:10:51 UTC | Merge pull request #129 from JuliaDiffEq/compathelper/new_version/2020-02-02-19-05-58-218-2470525580 CompatHelper: add new compat entry for "Optim" at version "0.20" | 02 February 2020, 19:10:51 UTC |
fa9715d | Christopher Rackauckas | 02 February 2020, 19:10:43 UTC | Merge branch 'master' into compathelper/new_version/2020-02-02-19-05-58-218-2470525580 | 02 February 2020, 19:10:43 UTC |
08088fa | Christopher Rackauckas | 02 February 2020, 19:10:16 UTC | Merge pull request #128 from JuliaDiffEq/compathelper/new_version/2020-02-02-19-05-52-170-3096553966 CompatHelper: add new compat entry for "Juno" at version "0.7" | 02 February 2020, 19:10:16 UTC |
1bc4e53 | github-actions[bot] | 02 February 2020, 19:06:06 UTC | CompatHelper: add new compat entry for "UnsafeArrays" at version "1.0" | 02 February 2020, 19:06:06 UTC |
777646d | github-actions[bot] | 02 February 2020, 19:06:01 UTC | CompatHelper: add new compat entry for "StaticArrays" at version "0.12" | 02 February 2020, 19:06:01 UTC |
2f509fa | github-actions[bot] | 02 February 2020, 19:05:58 UTC | CompatHelper: add new compat entry for "Optim" at version "0.20" | 02 February 2020, 19:05:58 UTC |
e5fce2c | github-actions[bot] | 02 February 2020, 19:05:53 UTC | CompatHelper: add new compat entry for "Juno" at version "0.7" | 02 February 2020, 19:05:53 UTC |
2bd7091 | Christopher Rackauckas | 02 February 2020, 18:29:10 UTC | Merge pull request #125 from JuliaDiffEq/sciml Introduce sciml_train | 02 February 2020, 18:29:10 UTC |
0973536 | Christopher Rackauckas | 02 February 2020, 18:28:22 UTC | Merge pull request #126 from JuliaDiffEq/fast implement fast versions of Flux | 02 February 2020, 18:28:22 UTC |
62fc7e2 | Chris Rackauckas | 02 February 2020, 17:06:58 UTC | fix fast_layers test | 02 February 2020, 17:06:58 UTC |
a17c292 | Chris Rackauckas | 02 February 2020, 17:01:09 UTC | fix up neural SDE in README | 02 February 2020, 17:01:09 UTC |
a0b6de8 | Chris Rackauckas | 02 February 2020, 11:46:38 UTC | Float32 default parameters | 02 February 2020, 11:46:38 UTC |
3622f80 | Chris Rackauckas | 02 February 2020, 10:59:18 UTC | support and use FastChain in neural SDE example | 02 February 2020, 10:59:18 UTC |
74543f7 | Chris Rackauckas | 02 February 2020, 10:33:41 UTC | initial_params interface to make it easier to swap out for Flux | 02 February 2020, 10:33:41 UTC |
e9149ce | Chris Rackauckas | 02 February 2020, 09:40:34 UTC | Static and unsafe (sounds like an awesome album name) | 02 February 2020, 09:40:34 UTC |
caad001 | Chris Rackauckas | 02 February 2020, 09:00:04 UTC | implement fast versions of Flux ```julia using DiffEqFlux, OrdinaryDiffEq, Optim, Flux, Zygote, Test u0 = Float32[2.; 0.] datasize = 30 tspan = (0.0f0,1.5f0) function trueODEfunc(du,u,p,t) true_A = [-0.1 2.0; -2.0 -0.1] du .= ((u.^3)'true_A)' end t = range(tspan[1],tspan[2],length=datasize) prob = ODEProblem(trueODEfunc,u0,tspan) ode_data = Array(solve(prob,Tsit5(),saveat=t)) fastdudt2,p = FastChain((x,p) -> x.^3, FastDense(2,50,tanh), FastDense(50,2)) fast_n_ode = NeuralODE(fastdudt2,p,tspan,Tsit5(),saveat=t) function fast_predict_n_ode(p) fast_n_ode(u0,p) end function fast_loss_n_ode(p) pred = fast_predict_n_ode(p) loss = sum(abs2,ode_data .- pred) loss,pred end dudt2 = Chain((x) -> x.^3, Dense(2,50,tanh), Dense(50,2)) n_ode = NeuralODE(dudt2,tspan,Tsit5(),saveat=t) function predict_n_ode(p) n_ode(u0,p) end function loss_n_ode(p) pred = predict_n_ode(p) loss = sum(abs2,ode_data .- pred) loss,pred end _p,re = Flux.destructure(dudt2) @test fastdudt2(ones(2),_p) ≈ dudt2(ones(2)) @test fast_loss_n_ode(p)[1] ≈ loss_n_ode(p)[1] @test Zygote.gradient((p)->fast_loss_n_ode(p)[1], p)[1] ≈ Zygote.gradient((p)->loss_n_ode(p)[1], p)[1] @btime Zygote.gradient((p)->fast_loss_n_ode(p)[1], p) @btime Zygote.gradient((p)->fast_loss_n_ode(p)[1], p) @btime Zygote.gradient((p)->loss_n_ode(p)[1], p) @btime Zygote.gradient((p)->loss_n_ode(p)[1], p) ``` ``` 27.272 ms (181318 allocations: 16.54 MiB) 27.328 ms (181318 allocations: 16.54 MiB) 262.430 ms (677868 allocations: 32.83 MiB) 260.814 ms (677868 allocations: 32.83 MiB) ``` order of magnitude performance improvement over using Flux for neural networks | 02 February 2020, 09:00:04 UTC |
f136e7c | Chris Rackauckas | 02 February 2020, 08:24:24 UTC | final sciml_train! README changes | 02 February 2020, 08:24:24 UTC |
db5291a | Chris Rackauckas | 02 February 2020, 05:57:49 UTC | fix up tests and most of README | 02 February 2020, 05:57:49 UTC |
aed5688 | Chris Rackauckas | 02 February 2020, 03:53:22 UTC | change partial_neural to new interface | 02 February 2020, 03:53:22 UTC |
d6a8422 | Chris Rackauckas | 02 February 2020, 03:38:22 UTC | standardize around Optim's output interface | 02 February 2020, 03:38:22 UTC |
150be4c | Chris Rackauckas | 02 February 2020, 02:05:04 UTC | start making the interface nicer | 02 February 2020, 02:05:04 UTC |
d057efb | Chris Rackauckas | 02 February 2020, 01:57:44 UTC | sciml passes | 02 February 2020, 01:57:44 UTC |
bb3da42 | Chris Rackauckas | 31 January 2020, 15:27:18 UTC | rename | 31 January 2020, 15:27:18 UTC |
3c9b00f | Chris Rackauckas | 31 January 2020, 15:24:37 UTC | Introduce sciml_train | 31 January 2020, 15:24:37 UTC |
2085732 | Christopher Rackauckas | 24 January 2020, 07:55:08 UTC | Update Project.toml | 24 January 2020, 07:55:08 UTC |
c51de86 | Christopher Rackauckas | 21 January 2020, 02:32:03 UTC | Merge pull request #117 from JuliaDiffEq/gpu_patch patch Zygote GPU support | 21 January 2020, 02:32:03 UTC |
91f21d7 | Chris Rackauckas | 20 January 2020, 21:46:39 UTC | set proper test brokens | 20 January 2020, 21:46:39 UTC |
b930c21 | Chris Rackauckas | 20 January 2020, 19:06:32 UTC | patch Zygote GPU support | 20 January 2020, 19:06:32 UTC |
8c7406d | Chris Rackauckas | 20 January 2020, 10:48:07 UTC | no longer broken with new recursivearraytools | 20 January 2020, 10:48:07 UTC |
22e20ad | Christopher Rackauckas | 20 January 2020, 10:16:25 UTC | Update Project.toml | 20 January 2020, 10:16:25 UTC |
bc97b36 | Christopher Rackauckas | 20 January 2020, 10:16:10 UTC | Merge pull request #115 from JuliaDiffEq/sde_dde NeuralDSDE, NeuralSDE, and NeuralCDDE | 20 January 2020, 10:16:10 UTC |
0a581a9 | Chris Rackauckas | 20 January 2020, 09:53:41 UTC | test disabled | 20 January 2020, 09:53:41 UTC |
aa5c064 | Chris Rackauckas | 20 January 2020, 09:24:13 UTC | for now | 20 January 2020, 09:24:13 UTC |
2777e74 | Chris Rackauckas | 20 January 2020, 08:10:30 UTC | easier test | 20 January 2020, 08:10:30 UTC |
c84759d | Chris Rackauckas | 20 January 2020, 06:05:20 UTC | remove requirement | 20 January 2020, 06:05:20 UTC |
192765f | Chris Rackauckas | 20 January 2020, 05:52:19 UTC | fix non-diagonal noise usage | 20 January 2020, 05:52:19 UTC |
d4f7a3d | Chris Rackauckas | 20 January 2020, 05:41:25 UTC | NeuralDSDE, NeuralSDE, and NeuralCDDE Removes NeuralDMSDE as part of the Zygote changes | 20 January 2020, 05:41:25 UTC |
f580921 | Chris Rackauckas | 20 January 2020, 03:56:31 UTC | Merge remote-tracking branch 'origin/master' | 20 January 2020, 03:56:31 UTC |
fe78dd1 | Chris Rackauckas | 20 January 2020, 03:56:27 UTC | Fix DMSDE Fixes https://github.com/JuliaDiffEq/DiffEqFlux.jl/issues/114 by making sure to do `re(p)` in the drift function call | 20 January 2020, 03:56:27 UTC |
bad3def | Christopher Rackauckas | 16 January 2020, 15:53:40 UTC | Update README.md | 16 January 2020, 15:53:40 UTC |
6d37feb | Christopher Rackauckas | 16 January 2020, 15:19:04 UTC | Update README.md | 16 January 2020, 15:19:04 UTC |
e2f0c14 | Christopher Rackauckas | 16 January 2020, 14:53:40 UTC | Update README.md | 16 January 2020, 14:53:40 UTC |
0c02e47 | Christopher Rackauckas | 16 January 2020, 14:02:35 UTC | Update README.md | 16 January 2020, 14:02:35 UTC |
04db452 | Christopher Rackauckas | 14 January 2020, 19:20:36 UTC | Update .gitlab-ci.yml | 14 January 2020, 19:20:36 UTC |
6f54e71 | Christopher Rackauckas | 14 January 2020, 18:23:22 UTC | update GPU to 1.3.1 | 14 January 2020, 18:23:22 UTC |
bfe3d91 | Christopher Rackauckas | 14 January 2020, 18:21:59 UTC | upstream fix to test | 14 January 2020, 18:21:59 UTC |
a5f431e | Christopher Rackauckas | 14 January 2020, 14:27:47 UTC | Update partial_neural.jl | 14 January 2020, 14:27:47 UTC |
6dbb90e | Christopher Rackauckas | 14 January 2020, 05:02:31 UTC | Update README.md | 14 January 2020, 05:02:31 UTC |
a1d4614 | Chris Rackauckas | 13 January 2020, 16:48:30 UTC | Merge remote-tracking branch 'origin/master' | 13 January 2020, 16:48:30 UTC |
2264c0a | Chris Rackauckas | 13 January 2020, 16:48:25 UTC | test broken the batched backsolve | 13 January 2020, 16:48:25 UTC |
96eb8d9 | Christopher Rackauckas | 13 January 2020, 14:09:53 UTC | Update Project.toml | 13 January 2020, 14:09:53 UTC |
0d81916 | Chris Rackauckas | 11 January 2020, 12:59:20 UTC | add a note on the new only time you'd need Tracker.collect | 11 January 2020, 12:59:20 UTC |
d9b373a | Chris Rackauckas | 11 January 2020, 12:56:37 UTC | little bit more clean up | 11 January 2020, 12:56:37 UTC |
333a757 | Chris Rackauckas | 11 January 2020, 12:53:08 UTC | No manual Tracker.collect ! | 11 January 2020, 12:53:08 UTC |
66516d7 | Chris Rackauckas | 10 January 2020, 13:27:46 UTC | some README clean up | 10 January 2020, 13:27:46 UTC |
5796081 | Chris Rackauckas | 10 January 2020, 06:12:29 UTC | Merge branch 'struct_layers' | 10 January 2020, 06:12:29 UTC |
6d9e0d4 | Chris Rackauckas | 10 January 2020, 06:12:17 UTC | trim down docs | 10 January 2020, 06:12:17 UTC |
8eb8659 | Christopher Rackauckas | 10 January 2020, 06:05:26 UTC | Merge pull request #109 from JuliaDiffEq/struct_layers Neural DE struct layers | 10 January 2020, 06:05:26 UTC |
d8501a5 | Chris Rackauckas | 10 January 2020, 06:01:20 UTC | fix up partial neural test | 10 January 2020, 06:01:20 UTC |
9185735 | Chris Rackauckas | 10 January 2020, 05:38:33 UTC | complete the transition to the new layers | 10 January 2020, 05:38:33 UTC |
9947c9c | Chris Rackauckas | 10 January 2020, 04:14:58 UTC | fix test | 10 January 2020, 04:14:58 UTC |
b89460b | Chris Rackauckas | 10 January 2020, 03:06:08 UTC | replace all layers with concrete_solve | 10 January 2020, 03:06:08 UTC |
61ebbe2 | Chris Rackauckas | 10 January 2020, 02:00:24 UTC | move all over to concrete_solve | 10 January 2020, 02:00:24 UTC |
851d7ff | Chris Rackauckas | 10 January 2020, 00:39:40 UTC | use as much concrete_solve as possible | 10 January 2020, 00:39:40 UTC |
4587a0a | Chris Rackauckas | 09 January 2020, 23:24:11 UTC | start updating all layers to concrete_solve | 09 January 2020, 23:24:11 UTC |
cda8235 | Chris Rackauckas | 09 January 2020, 22:46:13 UTC | change tests to the structs | 09 January 2020, 22:46:13 UTC |
1e0447a | Chris Rackauckas | 09 January 2020, 14:56:29 UTC | update README to use layer structs | 09 January 2020, 14:56:29 UTC |
f3f916b | Chris Rackauckas | 09 January 2020, 14:29:44 UTC | get it solving | 09 January 2020, 14:29:44 UTC |