Search code examples
pythonpython-2.7web-scrapingscrapyscrapy-item

has_key for item in Scrapy


I want to check if a field is set in a Scrapy item. But I use has_key on the item, I get this error:

Traceback (most recent call last):
  File "d:\python27\lib\site-packages\twisted\internet\defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "D:\Kerja\HIT\Python Projects\<project_name>\<project_name>\<project_name>\pipelines.py", line 16, in process_item
    self.listing_process.process_new_or_update(item)
  File "D:\Kerja\HIT\Python Projects\<project_name>\<project_name>\<project_name>\processor.py", line 290, in process_new_or_update
    listing = self.listing_check.normalize_for_process(listing)
  File "D:\Kerja\HIT\Python Projects\<project_name>\<project_name>\<project_name>\processor.py", line 213, in normalize_for_process
    if listing.has_key('description'):
  File "d:\python27\lib\site-packages\scrapy\item.py", line 74, in __getattr__
    raise AttributeError(name)
AttributeError: has_key

How can I check if the field is set without using has_key?


Solution

  • Finally found it. It turns out, even though we can use Scrapy on python 2.x, it expects us to use python 3 pattern. I should use 'field' in item instead of item.has_key('field').