Search code examples
pythonweb-scrapingbeautifulsoupurllib

incomplete result in table scraping


I am trying to scrape http://bifr.nic.in/asp/list.asp this page with beautifulsoup and get the table from it. Following is my code

from bs4 import BeautifulSoup
import urllib.request
base_url = "http://bifr.nic.in/asp/list.asp"

page = urllib.request.urlopen(base_url)
soup = BeautifulSoup(page, "html.parser")

table = soup.find("table",{"class":"forumline"})
tr = table.find_all("tr")
for rows in tr:
    print(rows.get_text())

It shows no error, but when i execute it i am only able to get first row of content from the table.

List of Companies

Case
            No
Company
            Name









 359  2000   A & F OVERSEAS LTD.





 359  2000   A & F OVERSEAS LTD.
 359  2000   A & F OVERSEAS LTD.

This is the result i am getting. I can't understand what's going on.


Solution

  • Try this to get all the data from that table:

    from urllib.request import urlopen
    from bs4 import BeautifulSoup
    
    page = urlopen("http://bifr.nic.in/asp/list.asp")
    soup = BeautifulSoup(page, "html5lib")
    table = soup.select_one("table.forumline")
    for items in table.select("tr")[4:]:
        data = ' '.join([item.get_text(" ",strip=True) for item in items.select("td")])
        print(data)
    

    Partial Output:

    359 2000 A & F OVERSEAS LTD.
    99 1988 A B C PRODUCTS LTD.
    103 1989 A INFRASTRUCTURE LTD.
    3 2006 A V ALLOYS LTD.
    13 1988 A V J WIRES LTD.