Search code examples
pythonpython-3.xscikit-learnpicklepipeline

unpickling model file python scikit-learn(Pipeline(memory=None, steps=None, verbose=None))


I am trying to convert pickle file from Python 2 to Python 3 using below code:

import os
import dill
import pickle
import argparse


def convert(old_pkl):
    """
    Convert a Python 2 pickle to Python 3
    """
    # Make a name for the new pickle
    new_pkl = os.path.splitext(os.path.basename(old_pkl))[0]+"_p3.pkl"

    # Convert Python 2 "ObjectType" to Python 3 object
    dill._dill._reverse_typemap["ObjectType"] = object

    # Open the pickle using latin1 encoding
    with open(old_pkl, "rb") as f:
        loaded = pickle.load(f, encoding="bytes")

    # Re-save as Python 3 pickle
    with open(new_pkl, "wb") as outfile:
        pickle.dump(loaded, outfile)

Pickling worked fine. But, the problem is when i tried to print the output of Python3 pickled file instead of showing below:

model = Pipeline([('count', CountVectorizer())
])

print(model)
Pipeline(memory=None,
     steps=[('count_vectorizer', CountVectorizer(analyzer='word', binary=False, decode_error='strict',
        dtype=<class 'numpy.int64'>, encoding='utf-8', input='content',
        lowercase=True, max_df=1.0, max_features=None, min_df=1,
        ngram_range=(1, 1), preprocessor=None, stop_words=None)])

it's showing below:

Pipeline(memory=None, steps=None, verbose=None) 

Solution

  • Found the solution:

    While unpickling the file i used encoding as bytes instead of latin1.

    Open the pickle using latin1 encoding

    with open(old_pkl, "rb") as f:
        loaded = pickle.load(f, encoding="latin1")
    

    and everything worked fine. For better clarification refer this