I am a bit new to machine learning and am trying to do linear regression without using linear_model.LinearRegression() via sklearn. I think I am nearing the end of my coding and getting prepared to plot the line but I am getting the error "ValueError: shapes (20,1) and (2,1) not aligned: 1 (dim 1) != 2 (dim 0)". I printed out my 20 by 1 matrices to confirm and neither of them have any extra dimensions or anything so I'm not sure why it is giving me (2,1) in the error message or why the dimensions don't match up. Does anyone have any recommendations on how to "align" these matrices? I am using Python 3.5.1.
Edit: I have looked at a lot of the other ValueError topics in stackoverflow but I am having trouble really understanding the recommendations. If possible, laymen's terms would be greatly appreciated.
Per Georgy I narrowed the code down to just the lines of code needed to cause the error message. alpha, iters, and theta were left alone to show all the variables required to be passed into the functions.
Edit 2: Alright, attempt 2 to reduce the example code. Thank you for working with me on this. I put a try-except statement around this line of code:
theta = theta -(alpha/len(X)) * np.sum((X @ theta.T - y) * X, axis=0)
Said line is nestled in a for loop using i as the variable. Said line is giving me the following error:
Traceback (most recent call last):
File "C:\Users\YungL\Desktop\linearRegression.py", line 30, in <module>
slope_and_intercept, cost = gradDescent(X_test, Y_test, theta, alpha, iters)
File "C:\Users\YungL\Desktop\linearRegression.py", line 26, in gradDescent
theta = theta -(alpha/len(X)) * np.sum((X @ theta.T - y) * X, axis=0)
ValueError: shapes (20,1) and (2,1) not aligned: 1 (dim 1) != 2 (dim 0)
Printing X[i], y[i], and theta when the exception is thrown gives me this respectively:
[[ 0.07786339] [[233.] [[1. 1.]]
Column 1 being X, column 2 being y, and column 3 being theta. Referencing the matrices, these are the 1st values in each matrix. Although for theta, that is the only value.
Here is the complete matrix of X and y at the time of the exception:
[[ 0.07786339] [[233.]
[-0.03961813] [ 91.]
[ 0.01103904] [111.]
[-0.04069594] [152.]
[-0.03422907] [120.]
[ 0.00564998] [ 67.]
[ 0.08864151] [310.]
[-0.03315126] [ 94.]
[-0.05686312] [183.]
[-0.03099563] [ 66.]
[ 0.05522933] [173.]
[-0.06009656] [ 72.]
[ 0.00133873] [ 49.]
[-0.02345095] [ 64.]
[-0.07410811] [ 48.]
[ 0.01966154] [178.]
[-0.01590626] [104.]
[-0.01590626] [132.]
[ 0.03906215] [220.]
[-0.0730303 ]] [ 57.]]
For matrix multiplication (which is what the @ operator does), you need the inner dimensions of the matrices in question to match. That is, you can multiply a 20 x 1 matrix by a 1 x 2 matrix, but not by a 2 x 1 matrix. This is not a numpy specific thing, it's just a basic fact of matrix arithmetic.
The issue you have is that X @ theta.T in your code is resulting in mismatched dimensions. I don't know what those variables represent (and you've edited the question to take out where they come from), but given the error, you probably want X @ theta instead. That will do the 20 x 1 and 1 x 2 multiplication, rather than the 2 x 1 multiplication that doesn't work mathematically.