Search code examples
pythonurllib2

Python urllib2 response

Okay so I'm currently sending a post request to a website which then outputs a response in html, I don't want to print the entire page, just specific content within some div brackets..

example of content i want to output

<td align="right"> example </td>

so I only want to output " example " within the td brackets, how would I do this?

Solution

  • I assume you are quite new to Python/programming in general.

    I recommend requests over built in urllib2 as it is easier to use.

    For element selection, I think beautifulsoup is one of the simplest libraries to use.

    Both are easy to install:

    1. pip install requests

    2. pip install beautifulsoup4

    Code:

    import requests 
from bs4 import BeautifulSoup
url = 'https://en.wikibooks.org/wiki/Unicode/List_of_useful_symbols'
r = requests.get(url)
soup = BeautifulSoup(r.text)
tds = soup.findAll("td") # you can extract tags like <div> as well
print(tds)
td_texts = [td.text for td in tds] # in case you are interested in the text only

    Output:

    [<td style="vertical-align:top">§</td>, <td> 00A7 section</td>, <td style="vertical-align:top">¶</td>, <td> 00B6 paragraph</td>, <td style="vertical-align:top">·</td>,