Search code examples
pythonbeautifulsoupurllib

Join BeautifulSoup Contents with map and lambda

I want to scrape the web contents and clean up the format

from bs4 import BeautifulSoup
import urllib.request
import urllib.parse
import lxml
url='https://en.wikipedia.org/wiki/Deep_learning'
page=urllib.request.urlopen(url)
soup=BeautifulSoup(page,"lxml")
fetched_text=' '.join(map(lambda p: p.text.soup.find_all('p'),soup))

It has an error like below: enter image description here

This code was originally found from a youtube tutorial like this:

fetched_text=' '.join(map(lambda p: p.text.soup.find_all('p')))

But it was complained about map() was not using correctly. enter image description here

Can anyone please help me out here?

Solution

  • Maybe you are more confortable using list comphension:

    fetched_text=' '.join([p.text for p in soup.find_all('p')])

    or you can do it with map, remember that map waits a function and an iterable:

    fetched_text=' '.join(map(lambda p: p.text, soup.find_all('p')))