how to solve problem with loading large amount of data in django?

Question

I'm trying to write an app like IMDb for my work project. In this app, I need to store 8700 IMDB movies and also about 40k directors and actors/actresses in MySQL database,

When I was trying to post a new movie, the movie creation page took a long time (about 20s) to render.

first, I made a model named "persons" with person name, gender, "is_director" and "is_actor" boolean fields.

then, I created a model named "movies" with 2 ManyToManyFields for is_director and is_actor persons.

after that, I imported about 40k persons to my database with Django fixture. my problem started from here. when I was trying to post a new movie from Django admin, the movie creation page took a long time (about 10s) to render. I know this is for a large amount of persons data. I did use Django built-in autocomplete but the problem was still there.

I'm new to Python/Django. so I think I'm doing this in a wrong way. maybe my database structure is wrong or my model has a problem, I don't know how to solve it!

# Person Model:
class Person(models.Model):
    name = models.CharField(_('Person Name'), max_length=100)
    is_actor = models.BooleanField(_('is actor'), default=True)
    is_director = models.BooleanField(_('is director'), default=False)
    MALE = 'male'
    FEMALE = 'female'
    PERSON_GENDER = [(MALE, _('Male')), (FEMALE, _('Female'))]
    gender = models.CharField(
        _('Gender'),
        max_length=6,
        choices=PERSON_GENDER,
        default=MALE
    )

# Movies Model:
class Movies(models.Model):
    title = models.CharField(_('Movie Name'), max_length=250)
    genre = models.ManyToManyField(Genre, verbose_name=_('genre'))
    director = models.ManyToManyField(
        Person, verbose_name=_('director'), related_name=_('director'),limit_choices_to={'is_director': True},blank=True)
    actors = models.ManyToManyField(
        Person, verbose_name=_('actors'), related_name=_('actors'),limit_choices_to={'is_actor': True},blank=True)

1- Is there a problem with my code?

2- Am I doing this in a wrong way?

3- Have I picked the wrong framework?

4- I'm Familiar with asp.net core and Laravel. Is it better to write this kind of app with them? language is no problem, I only need performance.

Answer 1

I guess your admin page contains your 40k persons list twice. Try to use raw_id_fields in the admin for your director and actors fields.

Please see Django documentation and this article about very long list (with an example with ForeignKey but it should also work with ManyToManyField)