Cannot extract URLs from a text file

Question

I'm trying to parse an online text file's contents then extract all URLs. Everything works fine except the URL extraction part. It doesn't happen. I tried the same process on a local file, and it worked. What is wrong?

COMMAND

import requests
import re
from io import StringIO

link = "https://pastebin.com/raw/B8QauiXU"
urls = requests.get(link)

with open(urls.text) as file, io.StringIO() as output:
    for line in file:
        urls = re.findall('https?://[^\s<>"]+[|www\.^\s<>"]+', line)
        print(*urls, file=output)

urls = output.getvalue()

print(urls)

OUTPUT

https://google.com and https://bing.com are both the two largest search engines in the world. They are followed by https://duckduckgo.com.

Answer 1

Making your regular expression a raw string works fine:

import requests, re
from io import StringIO

with StringIO() as output:
    link = "https://pastebin.com/raw/B8QauiXU"
    data = requests.get(link).text
    urls = re.findall(r'https?://[^\s<>"]+[|www\.^\s<>"]+', data)

    for i, url in enumerate(urls):
        output.write(f"{i}: {url}\n")
    print(output.getvalue())

Out:

0: https://google.com
1: https://bing.com
2: https://duckduckgo.com.