Search code examples
pythonrecursionpypdfpdfplumber

Scraping a sentence across many lines | Recursive error unresolved


Goal: if pdf line contains sub-string, then copy entire sentence (across multiple lines).

I am able to print() the line the phrase appears in.

Now, once I find this line, I want to go back iterations, until I find a sentence terminator: . ! ?, from the previous sentence, and iterate forward again until the next sentence terminator.

This is so as I can print() the entire sentence the phrase belongs in.

However, I have a Recursive Error with scrape_sentence() getting stuck infinitely running.


Jupyter Notebook:

# pip install PyPDF2
# pip install pdfplumber

# ---
# import re
import glob
import PyPDF2
import pdfplumber

# ---
phrase = "Responsible Care Company"
# SENTENCE_REGEX = re.pattern('^[A-Z][^?!.]*[?.!]$')

def scrape_sentence(sentence, lines, index, phrase):
    if '.' in lines[index] or '!' in lines[index] or '?' in lines[index]:
        return sentence.replace('\n', '').strip()
    sentence = scrape_sentence(lines[index-1] + sentence, lines, index-1, phrase)  # previous line
    sentence = scrape_sentence(sentence + lines[index+1], lines, index+1, phrase)  # following line    
    
    sentence = sentence.replace('!', '.')
    sentence = sentence.replace('?', '.')
    sentence = sentence.split('.')
    sentence = [s for s in sentence if phrase in s]
    sentence = sentence[0]  # first occurance
    print(sentence)
    
    return sentence
    
# ---    
    
with pdfplumber.open('../data/gri/reports/GPIC_Sustainability_Report_2020__-_40_Years_of_Sustainable_Success.pdf') as opened_pdf:
    for page in opened_pdf.pages:
        text = page.extract_text()
        lines = text.split('\n')
        i = 0
        sentence = ''
        while i < len(lines):
            if 'and Knowledge of Individuals; Behaviours; Attitudes, Perception ' in lines[i]:
                sentence = scrape_sentence('', lines, i)  # !
                print(sentence)  # !
            i += 1

Output:

connection and the linkage to the relevant UN’s 17 SDGs.and Leadership. We have long realized and recognized that there

Phrase:

Responsible Care Company

Sentence (across multiple lines):

"GPIC is a Responsible Care Company certified for RC 14001 
since July 2010."

PDF (pg. 2).


Please let me know if there is anything else I can add to post.


Solution

  • I solved this problem here by removing any recursion from scrape_sentence().