Symfony 5, memory leaking when send multiple emails through mailer

Question

I have memory leaking when sending bulk emails. I have around of 100 000 users to which I need to send every month newsletter. It's happen in production and in dev env as well.

I'm using:

• Symfony 5.1.11
• Docker
• Php 7.4.10

Below you can see the main piece of my command from execute method.

    // Total Users
    $totalRecords = $this->getUserRepository()->count(['newsletter' => true]);

    if ($totalRecords === 0) {
        $output->writeln('No records found');

        return Command::SUCCESS;
    }

    $offers = $this->fetchOffersByType($type);
    $totalOffers = count($offers);

    // Check if we have popular offers
    if ($totalOffers === 0) {
        $output->writeln('No Offers was found');

        return Command::SUCCESS;
    }

    $totalPages = ceil($totalRecords / self::BUFFER_SIZE);

    // Initializing one time and assign to users
    $newsletter = (new Newsletter())
        ->setSentAt(new DateTime())
        ->setType(NewsletterType::MAP[$type]);

    $totalSuccessSent = 0;
    $total = 0;
    for ($page = 1; $page <= $totalPages; $page++) {
        // Get users to who we will send newsletters
        $users = $this->getUserRepository()
            ->findBy(['newsletter' => true], null, self::BUFFER_SIZE, self::BUFFER_SIZE * ($page - 1));

        foreach ($users as $user) {
            $total++;
            if (empty($user->getEmail())) {
                continue;
            }

            if ($this->emailService->sendNewsletter($user, $type, $offers)) {
                $user->addNewsletter($newsletter);

                $this->em->persist($user);
                $totalSuccessSent++;
            }
        }

        $this->em->flush();

        // Make clean up after specific number of users
        if ($total % self::CLEAN_UP_AFTER === 0) {
            $output->writeln('Clean Up');
            $this->em->clear();
            gc_collect_cycles();
        }
    }

Basically I'm getting all users who are subscribed to newsletter and send 9 popular offers.

This piece is related to sendNewsletter.

     try {
        $email = (new TemplatedEmail())
            ->from(new Address($this->parameterBag->get('noReplayEmail'), $this->parameterBag->get('noReplayEmailName')))
            ->to($user->getEmail())
            ->priority(Email::PRIORITY_NORMAL)
            ->subject($subject)
            ->htmlTemplate($template)
            ->context([
                'offers' => $offers,
            ]);

        $this->mailer->send($email);

        return true;
    } catch (TransportExceptionInterface | RfcComplianceException | JsonException $e) {
        return false;
    }

After investigation I determined that $this->mailer->send($email) is the problem. This command has memory limit 512 and maximum +- 1500 emails it can send.

I'm using mailgun. The error can be different in some cases but this one happen more time:

php.CRITICAL: Fatal Error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 245760 bytes) {"exception":"[object] (Symfony\\Component\\ErrorHandler\\Error\\OutOfMemoryError(code: 0): Error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 245760 bytes) at /home/***/vendor/symfony/mime/Encoder/QpContentEncoder.php:36)"}

I also for test tried to use MAILER_DSN as null://null which means that it not will send the email. Same problem happen.

What else can I do ?

Update: Async sending of emails through Symfony Messages not helped me.

Answer 1

The problem was fixed in Symfony 5.2. Here is details of bug: https://github.com/symfony/symfony/pull/37712

So to all who have same problem, upgrade your symfony.