Search code examples
pythondictionarygeneratorkeyword-argument

Generator object to produce kwargs mapping for function


I have a generalised generator function, which maps delimited strings to a which yields a kwargs-type dictionary object from a list of delimited strings.

I'd like to use the kwargs-type dictionaries as arguments in another generator function. However, I get the message

edge_generator() argument after ** must be a mapping, not generator

Here is my current, broken code:

def line_from_file_generator(file_name):

    with open(file_name) as text_file:
        for line in text_file:
            yield line

def map_delimited_lines_to_kwargs(lines, delimiter, kwarg_index_map):

    for line in lines:

        kwargs = {}
        args = line.strip().split(delimiter)

        for kwarg, index in kwarg_index_map.items():
            kwargs[kwarg] = args[index]

        yield kwargs


def edge_generator(source_node_name, destination_node_name, weights):

    for line in lines:
        yield Edge(source_node_name, destination_node_name, weights)


lines = line_from_file_generator('mit_map.txt')
kwarg_index_map = {'source_node_name': 0, 'destination_node_name': 1, 'weights': 2}
kwarg_collection = map_delimited_lines_to_kwargs(lines, ' ', kwarg_index_map)
edges = edge_generator(**kwargs_collection)

I now understand why this is wrong (a generator is a kind of sequence, not a mapping (dict that can be expanded)).

Can I get around this whilst keeping the abstractions I have created? What would the code be? I've tried changing the last line to the below, but it fails to iterate through the kwargs generator, and all edges are generated with the same arguments.

edges = edge_generator(**kwarg_collection.next())


Solution

  • This is how I got it to work. Instead of trying to expand the arguments for the Edge generator in the function definition, I simply passed the kwargs generator whole, and added a loop over the kwargs generator in the body of the Edge generator, and expanded the arguments when creating the edge.

    def line_from_file_generator(file_name):
    
        with open(file_name) as text_file:
            for line in text_file:
                yield line
    
    
    def kwargs_from_delimited_strings_generator(lines, delimiter, kwarg_index_map):
    
        for line in lines:
    
            kwargs = {}
            args = line.strip().split(delimiter)
    
            for kwarg, index in kwarg_index_map.items():
                kwargs[kwarg] = args[index]
    
            yield kwargs
    
    
    def edge_generator(kwargs_collection):
    
        for kwargs in kwargs_collection:
            yield Edge(**kwargs)
    
    
    lines_from_file = line_from_file_generator('mit_map.txt')
    kwarg_index_map = {'source_node_name': 0, 'destination_node_name': 1,
                       'weights': 2}
    kwargs_from_delimited_strings = kwargs_from_delimited_strings_generator(
                                        lines_from_file, ' ', kwarg_index_map)
    edges_from_kwargs = edge_generator(kwargs_from_delimited_strings)