validated_data dropping some fields during POST requests on validation in .create(): nested serializer

Question

: purchased_products = validated_data.pop("products") KeyError: 'products'

I have a M2M relationship between Product and Purchase. What I am trying to achieve is when a purchase is made, to also fill the PurchasedProduct(the through model) model. But every time I send the data to the API and I try to access the products key in the serializer from the validated_data a keyError exception is thrown but if I return the validated_data for the purpose of debugging the product key is part of the response.

djangorestframework==3.11.0 django==2.2.10

class Product(Model):
    name = CharField(max_length=20, unique=True)
    date_added = DateTimeField(default=now)

models.py

class Purchase(Model):
    manager = ForeignKey('users.User', on_delete=PROTECT, related_name='purchases')
    quantity = DecimalField(max_digits=6, decimal_places=2)
    products = ManyToManyField('branches.Product', through='PurchasedProduct',
                               through_fields=('purchase', 'product'))
    amount_fc = IntegerField(default=0)
    amount_usd = IntegerField(default=0)
    total_amount = IntegerField(default=0)
    date_purchased = DateTimeField(default=now)


class PurchasedProduct(Model):
    purchase = ForeignKey('Purchase', on_delete=CASCADE, related_name="to_products", blank=True)
    product = ForeignKey('branches.Product', on_delete=CASCADE, related_name='purchases')
    unit_price = DecimalField(max_digits=12, decimal_places=4, default=0.00)
    quantity = DecimalField(max_digits=5, decimal_places=2)
    amount_fc = DecimalField(max_digits=10, decimal_places=2)
    date_purchased = DateTimeField(default=now)

serializer.py

class PurchasedProductSerializer(ModelSerializer):

    class Meta:
        model = PurchasedProduct
        fields = [
            "id",
            "purchase",
            "product",
            "unit_price",
            "quantity",
            "amount_fc",
            "date_purchased"
        ]


class PurchaseSerializer(ModelSerializer):
    # https://github.com/encode/django-rest-framework/issues/5403
    products = PurchasedProductSerializer(source="to_products", many=True)

    class Meta:
        model = Purchase
        fields = [
            "id",
            "manager",
            "quantity",
            "amount_fc",
            "amount_usd",
            "total_amount",
            "products",
            "date_purchased"
        ]

    def create(self, validated_data):
        purchased_products = validated_data.pop("products")
        manager = validated_data.pop('manager')
        quantity = validated_data.pop('quantity')
        amount_fc = validated_data.pop('amount_fc')
        amount_usd = validated_data.pop('amount_usd')
        total_amount = validated_data.pop('total_amount')

        purchase = Purchase.objects.create(
            manager=manager,
            quantity=quantity,
            amount_fc=amount_fc,
            amount_usd=amount_usd,
            total_amount=total_amount
        )

        for purchased_product in purchased_products:
            product = Product.objects.get(pk=purchased_product.pop("product"))

            purchase.products.add(product, through_default=purchased_product)
        return purchase

view.py

class PurchasesListView(ListCreateAPIView):
    queryset = Purchase.objects.all()
    serializer_class = PurchaseSerializer
    permission_classes = [AllowAny]
    filterset_fields = ['date_purchased', 'manager']

data

{
    "amount_fc": 13303340.0,
    "amount_usd": 1500,
    "manager": 2,
    "quantity": 100,
    "total_amount": 1230945,
    "products": [
        {
            "amount_fc": 1200334,
            "product": 8,
            "quantity": 120, 
            "unit_price": 10003.34
        },
        {
            "amount_fc": 1600334,
            "product": 6,
            "quantity": 100, 
            "unit_price": 16003.34
        }
    ]
}

Error:

purchased_products = validated_data.pop("products") KeyError: 'products'

But when I change purchased_products = validated_data.pop("products") to purchased_products = validated_data.pop("products", []) It works but it does not fill the through model(PurchasedProduct)

What I have tried

DRF example DRF doc exampe

drf-writable-nested drf-writable-nested does not support M2M with a through model

removing source products = PurchasedProductSerializer(many=True) added write_only=True read_only=False. I have also tried to suppress the uniqueValidator on the Product model but still does not work.

Answer 1

After a bit of googling plus some trial and error. I found two ways of solving this problem.

1. the first one is to get the key directly from self.context.

purchased_products = self.context['request'].data['products']

and it works fine. But I still don't understand why when trying to get products from validated_data it's throwing KeyError.

2. the second option is to use drf-nested. shout out to the team.🎊🎊🎉🎉

the package provides an implementation of .create() and .update() and many more for your serializer and the most important thing is they support M2M relationships with a through model. their implementation examples