Use DifferentiationInterface.jl

[1-Bart-1]

I would like to experiment with using FiniteDiff, as I expect it to be faster in my case. So I want to be able to provide an ADType which is then used both internally in ModelPredictiveControl.jl and JuMP.
https://github.com/JuliaDiff/DifferentiationInterface.jl?tab=readme-ov-file

[franckgaga]

Yes that's also in my TODO list!

In the last commits, I struggled to completely remove the memory allocations at runtime with ForwardDiff.jl (and without relying on StaticArray). It seems that DifferentiationInterface.jl implementation is cautious with the in-place differentiation APIs.

It's also possible to completely customize the AD that JuMP uses by changing the defaults here:

@operator(model, operator, dim, f[, ∇f[, ∇²f]])

Another advantage would be to provide the Hessian (or not, it would be an option). Right now the Hessian is skipped and Ipopt uses a BFGS approximation.

[1-Bart-1]

Why don't you want to use StaticArrays.jl? It should improve the speed of a lot of operations. They provide some kind of benchmark in their README.
https://github.com/JuliaArrays/StaticArrays.jl

[franckgaga]

Yeah nice bench!

But I think it's important to prioritize the built-in arrays of Julia. That's one of the main selling point of the language!

[franckgaga]

I could make the move to AbstractArray and AbstractVector although, to let the user decide.

[1-Bart-1]

Letting the user decide would be a good solution.

[gdalle]

In the last commits, I struggled to completely remove the memory allocations at runtime with ForwardDiff.jl (and without relying on StaticArray). It seems that DifferentiationInterface.jl implementation is cautious with the in-place differentiation APIs.

Hi @franckgaga, can you explain what you mean by this? Happy to help if necessary.

[franckgaga]

Hi @gdalle! I'm not using DifferentionInterface.jl at all right now in the package. I directly use ForwardDiff.jl.

This related to my question here on discourse:

https://discourse.julialang.org/t/getting-rid-of-forwarddiff-jacobian-allocations-when-using-closures/123939/7

I will continue the discussion on discourse for posterity.

[franckgaga]

@gdalle False alarm, I benchmarked and @code_warntype my Jacobian buffer and there is no allocations and type-instabilities at runtime.

I will push a slightly modified version of the buffer to prepare the gound for DifferentiationInterface.jl.

BTW, excellent work with this package, the API and the documentation is simple and clear 😄

ps: closure with many captured variables can get very mindf*** haha

[gdalle]

Hi there! Now that #171 is merged, what's the next step?

[franckgaga]

Not so much, the next release will probably integrate DI.jl.

About that @baggepinnen, the new default could be ForwardDiff.jl with the tracer sparsity detector, similar to the example here: https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/stable/tutorials/advanced/#Sparse-backends. Do you think it would be a good new default for nonlinear MPC ?

[baggepinnen]

Do you think it would be a good new default for nonlinear MPC ?

yes, if a sparse transcription is used. I don't think it would make sense as a default for single shooting.