Web Scraping Yahoo Finance attempt not working

Question

Thanks in advance for those who take thee time to read my issue!

I am trying to scrape some information off yahoo finance and have ran into some problems when I am running the program.

The Problem that I am running into is that in my last line when I am trying to write my values into the csv file, this error message keeps popping up:

Traceback (most recent call last): File "...", line 79, in csv_writer.writerow([j.name(), j.price(), j.pricetarget()]) TypeError: 'str' object is not callable

Basically it is this last line ( csv_writer.writerow([j.name(), j.price(), j.pricetarget(), j.findpg()]) that is running into the problem.

I am unsure whether it is that the particular values does not exist on the website. Why does my object appear as a string and hence render the code unable to call the functions of the objects? hmmm

Background info: stock tickers.txt is a file that I am extracting my stock ticker names in order to parse and find the respective pages and information on yahoo finance.

import requests
from bs4 import BeautifulSoup
import csv

csv_file = open('pricetarget.csv', 'w')
csv_writer = csv.writer(csv_file)
csv_writer.writerow(['Stock Ticker', 'Price', 'Price Target', 'Potential Gain'])

class ptfinder:
    def __init__(self, pturl, findpricetargettag, findpricetargetclass, priceurl, findpricetag, findpriceclass, nameurl, findnametag, findnameclass):
        self.pturl = pturl
        self.priceurl = priceurl
        self.findpricetag = findpricetag
        self.findpriceclass = findpriceclass
        self.nameurl = nameurl
        self.findnametag = findnametag
        self.findnameclass = findnameclass
        self.findpricetargettag = findpricetargettag
        self.findpricetargetclass = findpricetargetclass


    def name(self):
        self.source = requests.get(self.nameurl).text
        self.soup = BeautifulSoup(self.source, 'lxml')
        self.name = self.soup.find(self.findnametag, class_=self.findnameclass).text
        return self.name

    def price(self):
        self.source = requests.get(self.priceurl).text
        self.soup1 = BeautifulSoup(self.source, 'lxml')
        try:
            self.price = self.soup1.find(self.findpricetag, class_=self.findpriceclass).text
            return self.price
        except:
            print('h')
            return 'No Price Provided'

    def pricetarget(self):
        self.source = requests.get(self.pturl).text
        self.soup2 = BeautifulSoup(self.source, 'lxml')
        try:
            self.pricetarget = self.soup2.find(self.findpricetargettag, class_=self.findpricetargetclass).text
            return self.pricetarget
        except:
            return 'No Price Target Provided'


    def findpg(self):
        self.pg = str(((self.pricetarget() - self.price())/self.price())*100) + '%'
        return self.pg



stocksymbols = []
stockname = []
pt = []
textfile = open("/Users/ryanong/PycharmProjects/investing/stockksymbols.txt", 'r')
for line in textfile:
    shares = list(line.split(','))
    stocksymbols.append(shares[0])
    stockname.append(shares[1])

for i in range(50):
    checkstock = str(stocksymbols[i])
    nameurl = 'https://finance.yahoo.com/quote/' + checkstock + '/analysis?p=' + checkstock
    priceurl = 'https://finance.yahoo.com/quote/' + checkstock + '/analysis?p=' + checkstock
    pturl = 'https://finance.yahoo.com/quote/' + checkstock + '/analysis?p=' + checkstock
    s = ptfinder(
        pturl, 'span', "Trsdu(0.3s)",
        priceurl, 'span', 'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)',
        nameurl, 'h1', 'D(ib) Fz(18px)')
    pt.append(s)


for j in pt:
    csv_writer.writerow([j.name(), j.price(), j.pricetarget(), j.findpg()])

csv_file.close()

Answer 1

You're mixing method names and properties, for example:

    def pricetarget(self):                               # <--- method name is "pricetarget"
        self.source = requests.get(self.pturl).text
        self.soup2 = BeautifulSoup(self.source, "lxml")
        try:
            self.pricetarget = self.soup2.find(          # <--- here you're rewriting the method with string
                self.findpricetargettag, class_=self.findpricetargetclass
            ).text
            return self.pricetarget
        except:
            return "No Price Target Provided"

The solution is to rename def pricetarget(self) or the variable self.pricetarget.

Also, make sure that pricetarget() and price() return float and not string, because you will get error inside the findpg() function