Python - why is this a malformed node or string?

Question

I was just tidying up some code and ran into some trouble. I am getting an error I have never come across before

ValueError: malformed node or string:['0.000', '37.903', 'nosing']

My openLabels function at the top works fine and returns a list of items structured as you can see in the error message. I am doing some debugging and found it is the labelsToFrames function throwing the error. It is not accepting my list of lists as input. I have no idea why.

Any direction would be much appreciated!

def openLabels(pathLabels):
    path = (pathLabels + "*.lab")
    files = glob.glob(path)

    textCorpus = []
    for name in files:
        try:
            with open(name) as f:
                for line in f:
                    line = line.split()
                    textCorpus.append(line)

        except IOError as exc: 
            if exc.errno != errno.EISDIR:
                raise

    return textCorpus

def labelToFrames(labelCorpus):
    with labelCorpus as f_in:
        for line in f_in:
            song = ast.literal_eval(line)

        output = []                                 
        for block in song:
            block_start = float(block[0])           
            block_end = float(block[1])            
            singing = block[2]                      
            block_range = np.arange(block_start, block_end, 0.033)
        for x in block_range:
            ms_start = '{0:.3f}'.format(x)
            ms_end = '{0:.3f}'.format(x + 0.032)
            add_to_output = [ms_start, ms_end, singing]
            output.append(add_to_output)

        return(output)   


def main(): 
    pathLabels = "~/Train Labels/"
    labelCorpus = openLabels(pathLabels)
    labelCorpusFrames = labelToFrames(labelCorpus)

main()  


  File "<ipython-input-7-d1a356f3bed8>", line 1, in <module>
    labelCorpusFrames = labelToFrames(labelCorpus)

  File "<ipython-input-2-77bea44f1f3d>", line 54, in labelToFrames
    song = ast.literal_eval(line)

  File "*/lib/python3.6/ast.py", line 85, in literal_eval
    return _convert(node_or_string)

  File "*/lib/python3.6/ast.py", line 84, in _convert
    raise ValueError('malformed node or string: ' + repr(node))

ValueError: malformed node or string: ['0.000', '37.903', 'nosing']

Answer 1

The issue is that labelCorpus is a list of lists. As such, when you're executing labelToFrames and passing in labelCorpus, it is iterating over the list of lists, assigning each individual list to line and then trying to run ast.literal_eval on line. This then fails because ast.literal_eval requires either a string or expression node, not a list.

The reason that labelCorpus is a list of lists comes from when it is assigned a value from the openLabels function. In the following section, you iterate over the file paths returned by glob.glob and open the files:

with open(name) as f:
    for line in f:
        line = line.split()
        textCorpus.append(line)

On each open file you're iterating over the individual lines and then splitting each line in the file (which returns a list) and assigning it back to the line variable, which you then append to the textCorpus list. This list of lists is then what you return from the function and assign to labelCorpus in main.