Implementation of tanh() activation function for a CNN

Question

I'm trying to implement the activation function tanh on my CNN, but it doesn't work, the result is always "NaN". So i created a simple application where i have a randomized matrix and try to apply the tanh(x) function thus to understand where's the problem?

Here's my implementation :

    Eigen::MatrixXd A = Eigen::MatrixXd::Random(10,1000);
    Eigen::MatrixXd result, deriv;
    result = A.array().tanh();
    deriv = 1.0 - result*result;

and the only result to this is this error :

no match for ‘operator-’ (operand types are ‘double’ and ‘const Eigen::Product<Eigen::Matrix<double, -1, -1>, Eigen::Matrix<double, -1, -1>, 0>’)
     deriv = (1.0 - result*result );
              ~~~~^~~~~~~~~~~~~~~

Could you please help me ?

Answer 1

The product result*result does not have the right dimensions for a matrix multiplication. We can use result*result.transpose() instead (unless a coefficient-wise multiplication is intended, in which case one could use result.array()*result.array()).

To subtract the values of the resulting matrix from a matrix full of ones, the .array() method can be used:

deriv = 1. - (result*result.transpose()).array();