python-3.xnlptext-processingpos-tagger
How to insert pos tags in a dataframe arranged in separate columns in python?
I have POS tagged my input text using TextBlob and exported it into a text file. It gives me three information as POS, Parse Chunker and Deep Parsing. The output of this tagging is in this format: Technique:Plain/NNP/B-NP/O and/CC/I-NP/O . I want this to be arranged in a dataframe in separate columns for each.
This is the code I am using.
import pandas as pd
import csv
from textblob import TextBlob
with open('report1to8_1.txt', 'r') as myfile:
report=myfile.read().replace('\n', '')
out = TextBlob(report).parse()
tagS = 'taggedop.txt'
f = open('taggedop.txt', 'w')
f.write(str(out))
df = pd.DataFrame(columns=['Words', 'POS', 'Parse chunker','Deep
Parsing'])
df = pd.read_csv('taggedop.txt', sep=' ',error_bad_lines=False,
quoting=csv.QUOTE_NONE)
My expected result is to have a dataframe like this:
However, currently I am getting this:
Please Help!!
Solution
Try this. The example takes you through putting the data into the correct format so that you are able to create a dataframe. You need to create a list containing lists of your data. This data must be uniformly organised. Then you can create your dataframe. Comment if you need more help
from textblob import TextBlob as blob
import pandas as pd
from string import punctuation
def remove_punctuation(text):
return ''.join(c for c in text if c not in punctuation)
data = []
text = '''
He an thing rapid these after going drawn or.
Timed she his law the spoil round defer.
In surprise concerns informed betrayed he learning is ye.
Ignorant formerly so ye blessing. He as spoke avoid given downs money on we.
Of properly carriage shutters ye as wandered up repeated moreover.
Inquietude attachment if ye an solicitude to.
Remaining so continued concealed as knowledge happiness.
Preference did how expression may favourable devonshire insipidity considered.
An length design regret an hardly barton mr figure.
Those an equal point no years do. Depend warmth fat but her but played.
Shy and subjects wondered trifling pleasant.
Prudent cordial comfort do no on colonel as assured chicken.
Smart mrs day which begin. Snug do sold mr it if such.
Terminated uncommonly at at estimating.
Man behaviour met moonlight extremity acuteness direction. '''
text = remove_punctuation(text)
text = text.replace('\n', '')
text = blob(text).parse()
text = text.split(' ')
for tagged_word in text:
t_word = tagged_word.split('/')
data.append([t_word[0], t_word[1], t_word[2], t_word[3]])
df = pd.DataFrame(data, columns = ['Words', 'POS', 'Parse Chunker', 'Deep Parsing'] )
Result
Out[18]:
Words POS Parse Chunker Deep Parsing
0 He PRP B-NP O
1 an DT I-NP O
2 thing NN I-NP O
3 rapid JJ B-ADJP O
4 these DT O O
5 after IN B-PP B-PNP
6 going VBG B-VP I-PNP
7 drawn VBN I-VP I-PNP
8 or CC O O
9 Timed NNP B-NP O
10 she PRP I-NP O
11 his PRP$ I-NP O
12 law NN I-NP O
13 the DT O O
14 spoil VB B-VP O
15 round NN B-NP O
16 defer VB B-VP O
17 In IN B-PP B-PNP
18 surprise NN B-NP I-PNP
19 concerns NNS I-NP I-PNP
20 informed VBN B-VP I-PNP
21 betrayed VBN I-VP I-PNP
22 he PRP B-NP I-PNP
23 learning VBG B-VP I-PNP
24 is VBZ I-VP O
25 ye PRP B-NP O
26 Ignorant NNP I-NP O
27 formerly RB I-NP O
28 so RB I-NP O
29 ye PRP I-NP O
.. ... ... ... ...
105 no DT O O
106 on IN B-PP B-PNP
107 colonel NN B-NP I-PNP
108 as IN B-PP B-PNP
109 assured VBN B-VP I-PNP
110 chicken NN B-NP I-PNP
111 Smart NNP I-NP I-PNP
112 mrs NNS I-NP I-PNP
113 day NN I-NP I-PNP
114 which WDT O O
115 begin VB B-VP O
116 Snug NNP B-NP O
117 do VBP B-VP O
118 sold VBN I-VP O
119 mr NN B-NP O
120 it PRP I-NP O
121 if IN B-PP B-PNP
122 such JJ B-NP I-PNP
123 Terminated NNP I-NP I-PNP
124 uncommonly RB B-ADVP O
125 at IN B-PP B-PNP
126 at IN I-PP I-PNP
127 estimating VBG B-VP I-PNP
128 Man NN B-NP I-PNP
129 behaviour NN I-NP I-PNP
130 met VBD B-VP O
131 moonlight NN B-NP O
132 extremity NN I-NP O
133 acuteness NN I-NP O
134 direction NN I-NP O
[135 rows x 4 columns]