how to fix ''Found input variables with inconsistent numbers of samples: [219, 247]''

Question

As title says when running the following code i get a trouble Found input variables with inconsistent numbers of samples: [219, 247], i have read that the problem should be on the np.array set for X and y, but i cannot address the problem because there is a price for every date so i dont get why it is happening, any help will be appreciated thanks!

import pandas as pd
import quandl, math, datetime
import numpy as np
from sklearn import preprocessing, svm, model_selection
from sklearn.linear_model import LinearRegression
import matplotlib as plt
from matplotlib import style

style.use('ggplot')


df = quandl.get("NASDAQOMX/XNDXT25NNR", authtoken='myapikey')   
df = df[['Index Value','High','Low','Total Market Value']]
df['HL_PCT'] = (df['High'] - df['Low']) / df['Index Value'] * 100.0
df = df[['Low','High','HL_PCT']]

forecast_col = 'High'
df.fillna(-99999, inplace=True)

forecast_out = int(math.ceil(0.1*len(df)))

df['label'] = df[forecast_col].shift(-forecast_out)
df.dropna(inplace= True)

X = np.array(df.drop(['label'],1))

X = preprocessing.scale(X)

X_lately = X[-forecast_out:]

X = X[:-forecast_out]

y=np.array(df['label'])
#X= X[:-forecast_out+1]
df.dropna(inplace=True)
y= np.array(df['label'])

X_train, X_test, y_train, y_test= model_selection.train_test_split(X, 
y,test_size=0.2)

clf= LinearRegression(n_jobs=-1)
clf.fit(X_train, y_train)
accuracy = clf.score(X_test, y_test)
forecast_set= clf.predict(X_lately)
print(forecast_set, accuracy, forecast_out)
df['Forecast'] = np.nan

last_data= df.iloc[-1].name
last_unix= last_date.timestamp()
one_day=86400 
next_unix= last_unix + one_day

for i in forecast_set:
    next_date= datetime.datetime.fromtimestamp(next_unix)
    next_unix += one_day
    df.loc[next_date]= [np.nan for _ in range(len(df.columns) -1)] + 
[i]

df['High'].plot()
df['Forecast'].plot()
plt.legend(loc=4)
plt.xlabel('Date')
plt.ylabel('Price')
plt.show()

the expected result should be a plot of future price prediction for that ticker but besides that it is throwing that error 'Found input variables with inconsistent numbers of samples: [219, 247]'.

Answer 1

Your problem lies in these two lines extracted from your code:

X = X[:-forecast_out]
y= np.array(df['label'])

You're subsetting X, but leaving y "as it is".

You may check that shapes differ indeed by:

X.shape, y.shape

Change the last line to:

y= np.array(df[:-forecast_out]['label'])

and you're fine.

Note as well, instead of these repetitive lines:

y=np.array(df['label'])
#X= X[:-forecast_out+1]
df.dropna(inplace=True) # there is no na at this point
y= np.array(df['label'])

the following line (solution to your problem) is just enough:

y= np.array(df[:-forecast_out]['label'])