I am creating a web scraper application using C# and Selenium with Chrome WebDriver. So far I have configured it like this:
public class Program
{
private static ChromeOptions options;
private static ChromeDriver driver;
static void Main(string[] args)
{
var proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy = "127.0.0.1.fiddler:52840";
options = new ChromeOptions();
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
options.SetLoggingPreference(LogType.Driver, LogLevel.All);
driver = new ChromeDriver(options);
driver.Navigate().GoToUrl("https://www.google.com");
}
}
But I can't seem to get any info in Fiddler while accessing "https://www.google.com". I am also wondering - if I install an application on my PC (I am using Windows 10 Pro) which enables VPN while browsing, will this VPN be active while scraping or I need a manual configuration in the WebDriver?
By default fiddler will set Internet Options to use proxy and Chrome will follow it so there is no need to set it in your code. things to check:
Tools > Options > Connections > Fiddler listens on ports
127.0.0.1.fiddler
valid local domain or use local IP 127.0.0.1