Search code examples
scrapyjsonlines

JsonLinesItemExporter outputs an array in each field

I'm using JsonLinesItemExporter to export some data and instead of

{"name": "Color TV", "price": "1200"}
{"name": "DVD player", "price": "200"}

scrapy is writing the following to file:

{"name": ["Color TV"], "price": ["1200"]}
{"name": ["DVD player"], "price": ["200"]}

(From debug) it seems I'm passing a correct value (not a list) and that both item.add_value and item.replace_value are replacing my strings by a single string element list.

Is this configurable? If not, how to get a different behaviour? Extend JsonLinesItemExporter or is there a better approach?

Solution

  • Are you sure that you're using properly configured ItemLoader? I'de recommend to use TakeFirst (documentation here: https://docs.scrapy.org/en/latest/topics/loaders.html)

    Example of usage:

    class YourItemLoader(ItemLoader):
    default_output_processor = TakeFirst()