I'm trying to clear some tokens of the following punctuation by starting with an empty array then adding cleaned tokens to that array.
def main():
tokens = readTokens("words.txt")
depunctuatuate = depunctuateToken(tokens)
def readTokens(fileName):
s = Scanner(fileName)
items = []
token = s.readstring()
while (token != ""):
items.append(token)
token = s.readstring()
print(items)
s.close()
return items
def depunctuateToken(string):
result = []
s = Scanner(string)
token = s.readstring()
punct = [",",".","?",":","'","-"]
for i in range(0,len(string),1):
if(not(string[i] in punct)):
result += [string[i]]
return result
You don't need Scanner
in the depunctuateToken
function.
You are passing it an array of strings, Just iterate through them and do your punct
filtering on each one. You probably also want to modify your return statement, right now it looks like it will return the first item only.