Search code examples
pythonweb-scrapingscrapy

How do I give a body to a request in process_request?

This is the request I'm trying to give a body in the process_request method: yield scrapy.Request(url=self.url, method='POST', callback=self.parse)

This is how I tried to do it:

body = self.body.encode('utf-8') # I've tested the self.body, it contains the value I intended
request.replace(body=body)

But this is what I get: INFO: request body: b''

By the way, in the process_requests I return a None value, as far as I understand that shouldn't be the problem but, maybe I'm confusing something.

Solution

  • Request.replace() returns the new request object, it doesn't modify it in-place.