I want to write a simple gradient descent for a function with 5 parameters on C++. Now I stumble upon a implementation model problem: Should I sacrifice speed for folding my arguments in vector/arrays. Here is what I mean. I can implement funcition value and gradient calculation like this:
double function(double arg1, double arg2, ..., double arg5);
double functionGradient1(double arg1, double arg2, ..., double arg5);
double functionGradient2(double arg1, double arg2, ..., double arg5);
...
double functionGradient5(double arg1, double arg2, ..., double arg5);
or:
double function(double * args);
double functionGradientAt(double * args, int i);
The last one is easier to implement, however I am afraid I'll lose lots of speed if i am going to constantly allocate/free memory.
My questions are:
Which one would YOU pick?
I would pick:
double function(const vector<double> &args);
Or if it must be from your examples, then:
double function(double * args);
The reason being that I don't believe unrolling the arguments like that adds any noticeable speedup, but it does make the code look ugly.
If you ever implemented gradient descent, how did you handle the case?
I handled it with vectors.
Gradient descent's speed will probably depend more on the number of iterations you do than on how you pass these parameters. If you want to be as fast as possible, I would suggest against implementing it yourself and instead find a library that has it implemented. This question has a lot of good suggestions for that.