DiffRules [1.1,1.2] breaks precompilation

[abelsiqueira]

cf @mcabbott

More specifically, JuliaDiff/DiffRules.jl@e566656 breaks ForwardDiff precompilation. DiffRules 1.1 and 1.2 were released today, so I just noticed it on https://github.com/JuliaSmoothOptimizers/ADNLPModels.jl/pull/18/checks?check_run_id=2984684712#step:6:110.

On a fresh folder:

pkg> activate .
pkg> add DiffRules#e566656
pkg> add ForwardDiff
julia> using ForwardDiff

Fails with

[ Info: Precompiling ForwardDiff [f6369f11-7733-5829-9624-2563aa707210]
ERROR: LoadError: LoadError: ForwardDiff currently only knows how to autogenerate Dual definitions for unary and binary functions.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] top-level scope
    @ ~/.julia/packages/ForwardDiff/QOqCN/src/dual.jl:390
  [3] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
  [4] include(x::String)
    @ ForwardDiff ~/.julia/packages/ForwardDiff/QOqCN/src/ForwardDiff.jl:1
  [5] top-level scope
    @ ~/.julia/packages/ForwardDiff/QOqCN/src/ForwardDiff.jl:16
  [6] include
    @ ./Base.jl:386 [inlined]
  [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1213
  [8] top-level scope
    @ none:1
  [9] eval
    @ ./boot.jl:360 [inlined]
 [10] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [11] top-level scope
    @ none:1
in expression starting at /home/abel/.julia/packages/ForwardDiff/QOqCN/src/dual.jl:383
in expression starting at /home/abel/.julia/packages/ForwardDiff/QOqCN/src/ForwardDiff.jl:1
ERROR: Failed to precompile ForwardDiff [f6369f11-7733-5829-9624-2563aa707210] to /home/abel/.julia/compiled/v1.6/ForwardDiff/jl_fUdWWc.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1360
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1306
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1021
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901

[mcabbott]

Not >, the latest version is presently DiffRules 1.2.1, which should not break it. (For about an hour the latest was 1.1.0 or 1.2.0.)

So this is a bit of a surprise, and obviously it would have been ideal to test this package on top of those changes. But it seemed pretty innocuous to add N-ary rules when the docstring of @define_diffrule says "RHS should be the derivative expression, or in the n-ary case, an n-tuple of expressions". The PR was open since December.

It turns out that ForwardDiff doesn't merely not support such rules, but deliberately throws the error seen above if any are present: https://github.com/JuliaDiff/ForwardDiff.jl/blob/master/src/dual.jl#L390 . The obvious first step here would be not to do that, which should be an easy PR. (And to audit other packages using DiffRules to look for similar behaviour.)

I also think it might make sense to retrospectively add an upper bound at 1.0.2 to ForwardDiff's compat with DiffRules. I don't know if people have strong feelings about that, or whether it will cause subtle problems I haven't thought of. But I made JuliaRegistries/General#40225 so long.

[abelsiqueira]

Thanks, my tests were right during that window. I can confirm the precompilation passed for 1.2.1.

[andreasnoack]

So can this be closed?

[abelsiqueira]

For my needs, yes. Since there are more things related to this issue, I'll leave the closing to the maintainers' discretion.

[mcabbott]

I think this can be closed, with a newly updated registry you should get DiffRules v1.0.2 now, and never 1.1 or 1.2:

(jl_3xOmKz) pkg> add DiffRules, ForwardDiff
   Resolving package versions...
    Updating `/private/var/folders/yq/4p2zwd614y59gszh7y9ypyhh0000gn/T/jl_3xOmKz/Project.toml`
  [b552c78f] + DiffRules v1.0.2
  [f6369f11] + ForwardDiff v0.10.18