I'm crawling the website http://italianembassy.ir (which is only accessible through Iran's ip addresses) with selenium chrome webdriver using c#, but let me say sth about this site.
It is a very secured website against DDOS attacks and very strict about user activity, which blocks you immediately if you access it from two different systems with the same ip address(ex: two systems connected with one modem), and you need to change your ip address in order to access the site again(ex: restarting the modem).
Now, the problem with selenium here: I use the simplest code to start and stop the driver but any time I re-run my applicqation, im getting blocked with website, while I can open this site with chrome browser, close chrome, open it again and again without getting blocked.
var driver = new ChromeDriver();
driver.Navigate().GoToUrl("https://italianembassy.ir");
// do my crawling
driver.Quit();
I don't actually understand it. The chrome browser can access the site many times, but the chrome driver can do only once. When re-runnig my application, I get blocked(need to restart modem).
Do I miss sth about shutting down the driver safely??
Does selenuim lacks some capablities? Is it impossible to achive this due to the security applied to the site?( Im not good at network security)
I would be really glad if you help me solve this problem or give a logical reason about impossiblity of this operation.
Thanks in advance
Nice tip!
It worked for me, even better that the chrome itself
var driverOptions = new ChromeOptions();
driverOptions.AddArguments($"user-data-dir=C:/Users/{Environment.UserName}/AppData/Local/Google/Chrome/User Data/Default");
var driver = new ChromeDriver(driverOptions);
I mark this as solved But still wanna know what is behind it. What effects does the profile have on the behavior of web driver. I'd be glad to understand it.
Thanks again Madhan