Search code examples
djangoscreen-scraping

Want General Screen scraping application ( like facebook has ) for django


The use case of what I want is -
The user enters the link url ( like you might be doing in your facebook stat update box )
And a short description of this url with its title and a thumbnail appear.
( Yeah, the basic process is "sharing a link" )

How would you go about doing it ?
Call a django app which scrapes urls ( if yes then which one ? )
Do it using javascript? Is it possible ?


Solution

  • Have a look at BeautifulSoup, a very potent DOM crawler. With it, you could scrape URL's for tags and display them. The same goes for page titles.