Search code examples
python-2.7coremlturi

Turi Create - Please use dropna() to drop rows

I am having issues with Apple Turi Create and image classifier. I have successfully created a model with 22 categories. I have recently added 5 more categories and console is giving me error warning 

Please use dropna() to drop rows with missing target values.

The full console log looks like this:

[16:30:30] src/nnvm/legacy_json_util.cc:190: Loading symbol saved by previous version v0.8.0. Attempting to upgrade...
[16:30:30] src/nnvm/legacy_json_util.cc:198: Symbol successfully upgraded!
Resizing images...
Performing feature extraction on resized images...
Premature end of JPEG file
Completed  512/1883
Completed 1024/1883
Completed 1536/1883
Completed 1883/1883
PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.
      You can set ``validation_set=None`` to disable validation tracking.

[ERROR] turicreate.toolkits._main: Toolkit error: Target column has missing value.                    
Please use dropna() to drop rows with missing target values.
Traceback (most recent call last):
File "train.py", line 8, in <module>
model = tc.image_classifier.create(train_data, target='label', max_iterations=1000)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/turicreate/toolkits/image_classifier/image_classifier.py", line 132, in create
verbose=verbose)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/turicreate/toolkits/classifier/logistic_classifier.py", line 312, in create
seed=seed)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/turicreate/toolkits/_supervised_learning.py", line 397, in create
options, verbose)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/turicreate/toolkits/_main.py", line 75, in run
raise ToolkitError(str(message))
turicreate.toolkits._main.ToolkitError: Target column has missing value.                    

Please use dropna() to drop rows with missing target values.

I have upgraded turi and coremltools to the lates versions, but I don't know where I should implement the dropna() in the code. I only found this reference and followed the code.

It looks like this:

data.py

import turicreate as tc

image_data = tc.image_analysis.load_images('images', with_path=True)

labels = ['A', 'B', 'C', 'D']

def get_label(path, labels=labels):
  for label in labels:
       if label in path:
           return label

image_data['label'] = image_data['path'].apply(get_label)

#import os
#image_data['label'] = image_data['path'].apply(lambda path:    os.path.dirname(path).split('/')[-1])

image_data.save('boxes.sframe')

image_data.explore()

train.py

import turicreate as tc

data = tc.SFrame('boxes.sframe')
data.dropna()

train_data, test_data = data.random_split(0.8)

model = tc.image_classifier.create(train_data, target='label',    max_iterations=1000)

predictions = model.classify(test_data)

results = model.evaluate(test_data)

print "Accuracy           : %s" % results['accuracy']
print "Confusion Matrix   : \n%s" % results['confusion_matrix']

model.save('boxes.model')

How do I drop all the empty columns and rows please? Does the max_iterations=1000 have also effect on the error?

Thank you for suggestions

Solution

  • data.dropna() isn't done in place, you need to write it: data = data.dropna()
    See documentation here https://apple.github.io/turicreate/docs/api/generated/turicreate.SFrame.dropna.html