I am struggling to implement automatic differentiation in a plug-in library... can anybody help? No one has ever tried to add it at all, and it is a pain to find the derivative or the integral by hand.
The Deriv class in jlb.ulb does first and second derivatives of simple functions to a power, such as sin(z)^n. Other functions could be added.
These derivatives are courtesy of https://www.derivative-calculator.net/
But what if there was a way to apply the algorithms of automatic differentiation from https://github.com/autodiff/autodiff
in Ultra Fractal?Someone would need to write an UF equivalent which is definitely possible (albeit not with the nice syntax that you get in modern C++).
Ultra Fractal author
Then I had something in my head, a neuron activation of sorts: what if we used dual numbers, as well as nodes, forward mode and reverse mode autodifferentiation in UF?
So where do we begin, @ultrafractal ? First of all, we can think of a few ways to accomplish this: through nodes, for example.
I think I have found an easier way: dual numbers, and using the chain rule. Recall the Taylor expansion of f(x + y×epsilon):
We get two terms: f(x) + y×epsilon×(d/dx f(x)).
epsilon is any nilpotent basis other than 0 which returns 0 when squared.
The value y can be treated as a seed value, which can be set to 1. Thus, we get:f(x) + epsilon×(d/dx f(x)).
We can use real or complex values. When floats are used, the traditional dual number system is used with real numbers. When complex numbers are used, we get a four-dimensional system, the dual-complex numbers. The dual-complex numbers form a two-dimensional field over the complex numbers. Two classes can be created: Dual, and DualComplex. That is how its done, with forward and backward mode autodiff.
