Search code examples
c#seleniumselenium-chromedrivervpn

Track WebDriver requests with Fiddler or so


I am creating a web scraper application using C# and Selenium with Chrome WebDriver. So far I have configured it like this:

public class Program
{
    private static ChromeOptions options;
    private static ChromeDriver driver;

    static void Main(string[] args)
    {
        var proxy = new Proxy();
        proxy.Kind = ProxyKind.Manual;
        proxy.IsAutoDetect = false;
        proxy.HttpProxy = "127.0.0.1.fiddler:52840";
        
        options = new ChromeOptions();
        options.Proxy = proxy;
        options.AddArgument("ignore-certificate-errors");
        options.SetLoggingPreference(LogType.Driver, LogLevel.All);
        
        driver = new ChromeDriver(options);
        driver.Navigate().GoToUrl("https://www.google.com");
    }
}

But I can't seem to get any info in Fiddler while accessing "https://www.google.com". I am also wondering - if I install an application on my PC (I am using Windows 10 Pro) which enables VPN while browsing, will this VPN be active while scraping or I need a manual configuration in the WebDriver?


Solution

  • By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:

    1. Check Fiddler listen port: Tools > Options > Connections > Fiddler listens on ports
    2. make sure 127.0.0.1.fiddler valid local domain or use local IP 127.0.0.1
    3. Disable IPv6