Search code examples
performancematrixmatrix-multiplicationmultiplicationcomputation

How much computationally expensive is Matrix multiplication?

What advantage does the function

N(x;θ) = θ1(θ2*x)

has over

G(x;θ) = θ*x

for an input vector

x ∈ R^n
θ1 ∈ R^(nx1)
θ2 ∈ R^(1xn)
θ ∈ R^(nxn)
Solution

  • For first case, θ2 with dimension 1xn is multiplied with x with dimension n. That gives output of 1x1. Then multiplied by nx1 the output dimension of N(x;θ) is nx1. So there are n elements in θ2 and n elements in θ1. In total there are n+n (2n) elements.

    For second case, θ with dimension nxn is multiplied with x with dimension n. That gives output dimension for G(x;θ) as nx1. In this case there are n*n (n^2) elements for θ.

    Therefore, the advantage is that, it is computationally inexpensive to calculate the first case then the second case.