Automatic Differentiation in C# and F#

Question

I am having a problem getting Automatic Differentiation to work between C# and F#.

In C# I have a function that takes a double and returns a double, say:

private double Price(double yield)
{
    double price;

    price = 0;

    for (int index = 1; index <= _maturity * _frequency; index++)
    {
        price += (_coupon / _frequency) * _nominal / Math.Pow(1 + (yield / _frequency), index);
    }

    price += _nominal / Math.Pow(1 + (yield / _frequency), _maturity * _frequency);

    return price;
}

I picked this function specifically, as the Math.pow is very prohibitive, and only allows a double's or int's for its parameters.

I would like to differentiate this function using Automatic Differentiation. I have written the method for this in F#:

type Diff(d : double, df : Lazy<Diff>) = class
    member x.d = d
    member x.df = df
    static member (+) (x : Diff, y : Diff) = 
        Diff(x.d + y.d, lazy (x.df.Value + y.df.Value)) 
    static member (-) (x : Diff, y : Diff) = 
        Diff(x.d - y.d, lazy (x.df.Value - y.df.Value))
    static member (*) (x : Diff, a : double) = 
        Diff(x.d * a, lazy (x.df.Value * a))
    static member (*) (x : Diff, y : Diff) = 
        Diff(x.d * y.d, lazy ((x.df.Value * y) + (y.df.Value * x)))
    override x.ToString() =
        x.d.ToString()
end

let rec dZero = Diff(0.0, lazy dZero)

let dConst x = Diff(x, lazy dZero)

let dId x = Diff(x, lazy dConst 1.0)

let Differentiate (x:Diff) = x.df.Value

// Example function
let f (x:Diff) = x*x*x;

// Example usage:
// (f (dId 5)).ToString = "125"
// (Differentiate (f (dId 5))).ToString = "75"
// (Differentiate (Differentate (f (dId 5)))).ToString = "30"

Unfortunately, I need to feed a type Diff into my Price(..) function to produce a type Diff, which then gets fed into my Differente(..) function to return another type Diff.

My C# function however works solely on doubles (and I would like it to stay this way, as it is used in other places in my C# program).

The only way I can think to solve this is to write every function twice, which is obviously awful as:

1) I may as well just write a differentiated version each time 2) This isn't a very expandable model

So is there any way I can get around this, or perhaps coerce my double functions into Diff functions (preferably in F#). Ideally I would just like to throw a (double -> double) function in and get a Diff.ToString() out.

Sorry if this totally vague or impossible to understand. I will answer any questions in comments if this is unclear.

I hope there is a solution for this! Thanks in advance,

Ashley

Answer 1

You can re-invent Haskell Type Classes:

interface Eq<T>
{
    bool Equal(T a, T b);
    bool NotEqual(T a, T b);
}

interface Num<T> : Eq<T>
{
    T Zero { get; }
    T Add(T a, T b);
    T Subtract(T a, T b);
    T Multiply(T a, T b);
    T Negate(T a);
}

sealed class Int : Num<int>
{
    public static readonly Int Instance = new Int();
    private Int() { }
    public bool Equal(int a, int b) { return a == b; }
    public bool NotEqual(int a, int b) { return a != b; }
    public int Zero { get { return 0; } }
    public int Add(int a, int b) { return a + b; }
    public int Subtract(int a, int b) { return a - b; }
    public int Multiply(int a, int b) { return a * b; }
    public int Negate(int a) { return -a; }
}

Then you can do:

static T F<M, T>(M m, T x) where M : Num<T>
{
    return m.Multiply(x, m.Multiply(x, x));
}

static void Main(string[] args)
{
    Console.WriteLine(F(Int.Instance, 5));  // prints "125"
}

And then with:

class Diff
{
    public readonly double d;
    public readonly Lazy<Diff> df;

    public Diff(double d, Lazy<Diff> df)
    {
        this.d = d;
        this.df = df;
    }
}

class DiffClass : Floating<Diff>
{
    public static readonly DiffClass Instance = new DiffClass();
    private static readonly Diff zero = new Diff(0.0, new Lazy<Diff>(() => DiffClass.zero));
    private DiffClass() { }
    public Diff Zero { get { return zero; } }
    public Diff Add(Diff a, Diff b) { return new Diff(a.d + b.d, new Lazy<Diff>(() => Add(a.df.Value, b.df.Value))); }
    public Diff Subtract(Diff a, Diff b) { return new Diff(a.d - b.d, new Lazy<Diff>(() => Subtract(a.df.Value, b.df.Value))); }
    public Diff Multiply(Diff a, Diff b) { return new Diff(a.d * b.d, new Lazy<Diff>(() => Add(Multiply(a.df.Value, b), Multiply(b.df.Value, a)))); }
    ...
}

You can do this:

static T Price<M, T>(M m, T _maturity, T _frequency, T _coupon, T _nominal, T yield) where M : Floating<T>
{
    T price;

    price = m.Zero;

    for (T index = m.Succ(m.Zero); m.Compare(index, m.Multiply(_maturity, _frequency)) <= 0; index = m.Succ(index))
    {
        price = m.Add(price, m.Divide(m.Multiply(m.Divide(_coupon, _frequency), _nominal), m.Power(m.Add(m.Succ(m.Zero), m.Divide(yield, _frequency)), index)));
    }

    price = m.Add(price, m.Divide(_nominal, m.Power(m.Add(m.Succ(m.Zero), m.Divide(yield, _frequency)), m.Multiply(_maturity, _frequency))));

    return price;
}

But that's not really pretty.

In fact, it almost reads like code that creates a LINQ Expression Tree. Maybe you can use Source code Expression tree transformation instead of Operator overloading to achieve Automatic differentiation?