Search code examples
pythonpandasscikit-learnsklearn-pandas

Loading SKLearn 20_newsgroups dataset into Pandas DataFrame


Python: I'm trying to load a sklearn.20_newsgroups dataset sklearn.utils.Bunch into pandas dataframe.

I downloaded datasets the below link

categories = ["alt.atheism", "alt.atheism" ,"comp.os.ms-windows.misc" , "comp.sys.ibm.pc.hardware",
                "comp.sys.mac.hardware" , "comp.windows.x","misc.forsale", "rec.autos","rec.motorcycles",
                "rec.sport.baseball","rec.sport.hockey", "sci.crypt","sci.electronics", "sci.med","sci.space",
                "soc.religion.christian","talk.politics.guns" ,"talk.politics.mideast","talk.politics.misc" ,"talk.religion.misc"]

docs_to_train = sklearn.datasets.load_files("/home/Documents03-04-2019/dataset/20_newsgroups", 
                                      description    = None, 
                                      categories     = categories,
                                      load_content   = True,
                                      encoding       = 'ISO-8859-1',
                                      shuffle        = True,
                                      random_state   = 42)

The below code I treid.

docs_to_train.keys()
data1           = pd.DataFrame(docs_to_train.data, columns=docs_to_train.target_names])
data1['Target'] = pd.Series(data1=docs_to_train.target, index=data1.index)

Desired output I ran the below similar kind of code its working similarly i need newsgroups like a dataframe format.

from sklearn.datasets import load_breast_cancer
data = pd.DataFrame(cancer.data, columns=[cancer.feature_names])
data['Target'] = pd.Series(data=cancer.target, index=data.index)

Solution

  • Several of your keywords reference unrelated code: you write cancer or data instead of data1, and there's an unmatched ].

    Try this:

    data1 = pd.DataFrame(docs_to_train.data, columns=[docs_to_train.target_names])
    data1['Target'] = pd.Series(data=docs_to_train.target, index=data1.index)
    

    If that doesn't work, try this instead of the second line:

    data1['Target'] = pd.Series(data=docs_to_train.target)