Search code examples
pythonloopsweb-scrapingbeautifulsoupcontrol-flow

Loop the following code until if statement is True


Having trouble looping this code until the correct keywords are found, only then should this code continue executing past the comment line... As of now this code continues past the comment line whether the given keywords are found or not.

from bs4 import BeautifulSoup
import requests
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import urllib2
import time
from lxml import etree
while True:
    keyword1 = "example"
    keyword2 = "stackoverflow"
    keyword3 = "notactuallyonwebsite"
    print("starting")
    r = requests.get('http://kithnyc.com/sitemap_products_1.xml?from=60594372&to=9545825095')
    soup = BeautifulSoup(r.text, 'lxml')
    links = soup.find_all('loc')
    for link in links:
        if keyword1 in link.text and keyword2 in link.text and keyword3 in link.text:
            logic = True
        if logic == True:
            continue
    #################################### comment line ####
    print(link.text)
    jake = str(link.text)
    print(jake + "link scraped")

Solution

  • The continue applies to the inner loop, not the outer one. Also, you don't need to set a variable and then check it; just check the condition itself.

    links = soup.find_all('loc')
    if all(keyword1 in link.text and keyword2 in link.text for link in links):
        continue
    #################################### comment line ####