Search code examples
pythonweb-scrapingbeautifulsoupweb-crawler

find function in Beautifulsoup return None in first list


I now practice to parse HTML with Beautifulsoup4. I encounter problems using find function. Here is my code.

soup1 = BeautifulSoup(a,"html.parser")
tables1 = soup1.find('div', {'id':'auction_container'}).findAll('table')
for table in tables1:
if '매각기일' in table.get_text():
    clue1 = table.find('td', {'class': 'head_con center'})
    pro_clue1 = clue1.find('span', {'class':'bold'})
    pro_clue2 = clue1.find('span',{'class':'no'})
    clue2 = table.find('tr', {'valign': 'bottom'})
    print(clue2.find('span', {'class': 'num'}))

variable a is too long pagesource, so I write full script in my blog. You can get the script in this like. http://blog.naver.com/khm2963/220983094160 When I execute this code, I got output below

None
<span class="num"><span class="f20">2015</span>타경<span class="f20">2321</span></span>

And When I append .get_text () function behind clue2.find('span', {'class': 'num'}) like print(clue2.find('span', {'class': 'num'}).get_text()), I got error below.

Traceback (most recent call last):
File "D:/python code/auction_crawl/test bs4.py", line 5895, in <module>
print(clue2.find('span', {'class': 'num'}).get_text())
AttributeError: 'NoneType' object has no attribute 'get_text'

If I print print(clue2) without .find('span', {'class': 'num'}) I got result like below

<tr valign="bottom">
<td class="head_num left"><img alt="굿옥션로고" height="26" 
src="/img/common/top_logo.gif" width="100"><span class="logo_pid"></span>
</img>
</td>
<td class="head_con center">
<span class="bold">서울남부지방법원  본원 8계(02-2192-1338)</span> / 매각기일 : 
<span class="bold"><span class="no">2017.04.12(水)</span> <span class="no">
(10:00)</span>
</span></td>
</tr>
<tr valign="bottom">
<td class="head_num bold no left" style="width:190px;padding:10px 0 2px 
0;font-size:15px"><span class="num"><span class="f20">2015</span>타경<span 
class="f20">2321</span></span></td>
<td class="head_con center" style="padding-bottom:6px"><div>
<span class="ltblue"><img src="/img/icon/point_blue.gif" style="vertical-
align:middle"/></span> <span class="blue bold">서울남부지방법원  본원
</span>  <span class="ltblue"><img src="/img/icon/point_blue.gif" 
style="vertical-align:middle"/></span> 매각기일 : <span class="blue bold 
no">2017.04.12(水) (10:00)</span>  <span class="ltblue"><img 
src="/img/icon/point_blue.gif" style="vertical-align:middle"/></span> <span 
class="blue bold">경매 8계</span>(전화:02-2192-1338)</div>
</td>
</tr>

So I made HTML code above to variable d. and made another code like below.

d = ''' HTML code above  '''
soup4= BeautifulSoup(d,"html.parser")
clue = soup4.find('span', {'class': 'num'})
print(clue.get_text().strip())

When I activate the code above, I got response like this 2015타경2321. This is what I want. I want to get 2015타경2321 from the top code. How can I get it??


Solution

  • You can just verify if your clue2.find('span', {'class': 'num'}) has results and if so, print the result:

    ...
    clue2number = clue2.find('span', {'class': 'num'})
    if clue2number is not None:
        print (clue2number.get_text(strip=True))
    

    Which outputs:

    2015타경2321