ValueError: could not convert string to float: id

Question

I'm running the following Python script:

#!/usr/bin/python

import os,sys
from scipy import stats
import numpy as np

f = open('data2.txt', 'r').readlines()
for i in range(0, len(f)-1):
    l1 = f[i].split()
    list1 = [float(x) for x in l1]

However I got the errors like:

ValueError: could not convert string to float: id

I'm confused by this.

When I try this for only one line in interactive section, instead of for loop using script:

from scipy import stats
import numpy as np

f = open('data2.txt','r').readlines()
l1 = f[1].split()
list1 = [float(x) for x in l1]
list1
# [5.3209183842, 4.6422726719, 4.3788135547]

it works well. What is the explanation a little bit about this?

Answer 1

Obviously some of your lines don't have valid float data, specifically some line have text id which can't be converted to float.

When you try it in interactive prompt you are trying only first line, so best way is to print the line where you are getting this error and you will know the wrong line e.g.

#!/usr/bin/python

import os,sys
from scipy import stats
import numpy as np

f=open('data2.txt', 'r').readlines()
N=len(f)-1
for i in range(0,N):
    w=f[i].split()
    l1=w[1:8]
    l2=w[8:15]
    try:
        list1=[float(x) for x in l1]
        list2=[float(x) for x in l2]
    except ValueError,e:
        print "error",e,"on line",i
    result=stats.ttest_ind(list1,list2)
    print result[1]