Search code examples
pythonbeautifulsoupscreen-scraping

Finding articles under a specific ID using Beautifulsoup


Here is a screenshot of the section I want to web scrape, I took a screenshot because I just wanted to give an overall picture

Im trying to web scrape a certain section of the BBC site, this is the latest updates section. It looks a bit like this enter image description here

Im not sure what to put for the findAll in Beautifulsoup. Using the knowledge I already have I tried bsObj.findAll(id="latest-updates") which of course didn't work. If anyone needs a link of the latest update it would be here https://www.bbc.co.uk/news/uk and scrolling down gives your the latest update

Im trying to find a way to obtain all the links in the latest update section by using the id perhaps? If the correct way is using id=latest-updates, could someone point me to what I did wrong - or else if Im completely wrong tell me which class, id or div Im supposed to find the links under this specific latest update I should be looking for.
Sorry if this is too much of a specific question and
Thanks for any help regarding the question


Solution

  • If you're looking for the links of the headlines of latest updates, i think you should search for "a" and the class of the headlines. If I'm not wrong, this should be the right findAll:

     findAll("a", {"class": "qa-heading-link lx-stream-post__header-link"})