Why AttributeError: 'bytes' object has no attribute 'findAll'

Question

I am trying to screape the youtube data from trending page. Got error

from bs4 import BeautifulSoup
import requests
import csv

source = requests.get("https://www.youtube.com/feed/trending").text
soup = BeautifulSoup(source, 'lxml').encode("utf-8")

csv_file = open('YouTube.csv','w')
csv_writer = csv.writer(csv_file)
csv_writer.writerow(['Title', 'Description'])

for content in soup.findAll('div', class_= "yt-lockup-content"):
#for content in soup.find_all('div', class_= "yt-lockup-content"):
    print (content)

Answer 1

The AttributeError: 'bytes' object has no attribute 'findAll' is because in your code you are doing:

soup = BeautifulSoup(source, 'lxml').encode("utf-8")

What is happening is, you are converting str to byte by encoding the string. encode is used to convert str to bytes with the encoding of choice.

One should never be manipulating with bytes inside the program. Instead use the unicode sandwitch principle. Which is convert bytes to str on read, do stuff with str, then convert str to bytes on write to output.

So just use the str inside the program, instead of bytes like,

soup = BeautifulSoup(source, 'lxml')