Search code examples
c#performanceseleniumphantomjsscreen-scraping

Selenium and PhantomJS takes 30 seconds to open each link


I am trying to open a website and grab some data using Selenium with PhantomJS, however it takes a lot of time to open a website (about 30 second). And every time I open other link I have to wait 30+Seconds. What is wrong with my code ?

        static void Main(string[] args)
        {
        IWebDriver browser = new PhantomJSDriver();

        var URL = "http://www.cbssports.com/nba/playerrankings ";

        browser.Navigate().GoToUrl(URL);

        //Position
        var title = browser.FindElements(By.CssSelector(".tableTitle"));
        Console.WriteLine(title.First().Text);

        Console.Read();
        }

Things I have tried to do:
1.Set PhantomJS proxy type to none
2.Disable internet option: automatically detect settings
3.Disable IPv6 protocol

PhantomJS release notes claim, that there are some known issues with network performance on Microsoft Windows. According to release notes, solution is to set proxy type to none, however that doesn't work.


Solution

  • You have to wait 30 seconds because you haven't defined timeouts which are 30 seconds as default. You should use this predefined driver service.

            var phantomJSDriverService = PhantomJSDriverService.CreateDefaultService();
            IWebDriver browser = new PhantomJSDriver(phantomJSDriverService);
            browser.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(0));