I'm using selenium grid in docker. I want to login to website that I want to scrape before run in parallel many tasks. In general I want do login then keep auth data and share between all my nodes. Is it possible or maybe there is some nicer way to do this? Because when I run in parallel all task I have no auth data, and I need to login on each
My compose:
services:
selenium-hub:
image: selenium/hub:3.141.59-20210311
container_name: selenium-hub
environment:
GRID_MAX_SESSION: 16
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
chrome:
image: selenium/node-chrome:3.141.59-20210311
volumes:
- /dev/shm:/dev/shm
depends_on:
- selenium-hub
environment:
HUB_HOST: selenium-hub
HUB_PORT: 4444
NODE_MAX_SESSION: 5
NODE_MAX_INSTANCES: 5
I didn't find how to reuse session, but I found how to do it yourself. First of all I do login and then save cookies
# login logic...
cookies = driver.get_cookies()
After that I can setup these cookies to all my nodes.
# Before setup your cookies go to related doamin. Without this cookies will not be setup
for cookie in cookies:
driver.add_cookie(cookie)