Selenium - Wait for the webpage to request external URL

Question

I am basically trying to get a link to a m3u8 file. When the page first loads it requests the m3u8 file with a secret token that is changed everyday. The file is downloaded when the page if fully loaded. I can see it using chrome developer tools at network tab. Since this token is generated dynamically I need to let the website request this file first, then give me the URL to the file including the token (something like http://secret-website.com/file?token=342782g1bud1).

I have never used selenium, so I am wondering how I can do this if possible. I need to do it programmatically using python or c#.

Answer 1

I've found a solution which will wait untill it finds the .m3u8 file. I am using fiddlecore as proxy so I can see what is being sent by the browser. This will let me capture any request, and if the request contains .m3u8 it will simply print out the URL (which I need). Note that this won't work with https as fiddlecore needs certifications for that (I guess it is easy to fix). This is a simple working code.

   bool close = false;

    // - Initialize fiddler for http proxy

    Fiddler.FiddlerApplication.Startup(5000, true, false);
    Fiddler.FiddlerApplication.BeforeResponse += delegate(Session s)
    {
        string path = s.fullUrl;
        if (path.Contains("720p.m3u8"))
        {
            Console.WriteLine(path);
            close = true;
        }
    };

    // - Create a selenium proxy
    var seleniumProxy = new OpenQA.Selenium.Proxy { HttpProxy = "localhost:5000"};
    var profile = new FirefoxProfile();
    profile.SetProxyPreferences(seleniumProxy);

    // - Initialize selenium for browser automation
    IWebDriver driver = new FirefoxDriver(profile);
    driver.Navigate().GoToUrl("http://www.asite");
    while (!close) { }

    driver.Quit();
    Fiddler.FiddlerApplication.Shutdown();