django-import-export - Importing ManyToMany from JSON file

Question

I'm trying to import a JSON file that contains a many-to-many relationship but am hitting some problems. This is my code:

book.json

{
  "title": "The Lord of the Rings",
  "categories": [
      "Fantasy",
      "Action",
      "Adventure"
  ]
}

models.py

from django.db import models

class Category(models.Model):
    guid = models.CharField(max_length=36)
    display_name = models.CharField(max_length=50)
    description = models.TextField()

    def __str__(self):
        return self.display_name

class Book(models.Model):
    title = models.CharField(max_length=200)
    categories = models.ManyToManyField(Category)

    def __str__(self):
        return self.title

resources.py

from import_export import resources
from import_export.fields import Field
from import_export.widgets import ManyToManyWidget
from .models import Book, Category

class BookResource(resources.ModelResource):
    categories = Field(widget=ManyToManyWidget(Category))

    class Meta:
        model = Book
        import_id_fields = ('title',)

importer.py

from resources import BookResource
from models import Book

with open("book.json", 'r') as f:
        book = '[' + f.read() + ']'
        result = BookResource().import_data(tablib.Dataset().load(book), raise_errors=True, dry_run=True)

This seems to run through without errors, and adds the new book to the database. The problem is that it doesn't create a new entry in the book_categories table (PostgreSQL). I think I'm definitely missing a step somewhere because I haven't defined how to find the correct category database entries, given the list of display_name strings. Am I on the right track here?

Answer 1

So the short story is, that this case isn't handled by django-import-export: if you look at ManyToManyWidget.clean(), you see that it expects a string of comma separated integers by default. There's a bit you can do there, but it simply wants you to import the Categories first and then the Books.

What I ended up doing was remove the Field and Widget on BookResource and replace it with the widgets dictionary:

class BookResource(resources.ModelResource):
    class Meta:
        model = Book
        import_id_fields = ("title",)
        fields = ("id", "title", "categories")
        widgets = {"categories": {"field": "display_name"}}

If you then import Categories first, then your books will be correctly linked. Well, with one modification to the JSON and I wrapped the importer script up, so it can be run on console:

import django
import os

os.environ["DJANGO_SETTINGS_MODULE"] = "testbed.settings"

BOOK_DATA = """[{
  "title": "The Lord of the Rings",
  "categories": "Fantasy,Action,Adventure"
}]"""


def run():
    from bookstore.resources import BookResource
    import tablib

    importer = BookResource()
    dataset = tablib.Dataset()
    data = dataset.load(BOOK_DATA, format="json")
    result = importer.import_data(data, raise_errors=True, dry_run=False)
    print(result.has_errors())


if __name__ == "__main__":
    import sys

    sys.path.insert(0, os.path.abspath(os.path.dirname(os.path.dirname(__file__))))
    django.setup()
    run()

It is possible to write a JSON specific M2M widget, but then when importing categories, you will need to figure out each row which already exist and which don't. It's doable, so maybe I've put you on the right track to fix it ;)