RSelenium Enter User details in prompt

Question

I use RSelenium to connect to an internal website that asks for user login via a prompt like this:

(note that the URL=red bar is redacted)

Connecting to the Selenium Server and accessing the website works, but I cannot access the user prompt and therefore cannot login.

Is there any way I can provide user and pw to the prompt using RSelenium?

I use the code like so:

Note that for testing purposes you can use this URL: https://the-internet.herokuapp.com/basic_auth with admin admin.

library(RSelenium)
sel <- list(addr = "THE-SELENIUM-IP", port = 4444)

dr <- remoteDriver(
  remoteServerAddr = sel$addr,
  port = sel$port
)
res <- dr$open()

full_url <- "https://the-internet.herokuapp.com/basic_auth"
dr$navigate(full_url)

using dr$screenshot(display = TRUE) I receive this error Dismissed user prompt dialog: This site is asking you to sign in

Alternatively, I can try to define User and PW in the url like so:

user <- "MYUSER"
pw <- "MYPASSWORD@123"

full_url <- paste0("http://", user, ":", URLencode(pw, reserved = TRUE), "@https://the-internet.herokuapp.com/basic_auth")

dr$navigate(full_url)
dr$screenshot(display = TRUE)

But I get the same error.

Note that this is a prompt and not part of the HTML website (ie I cannot findElementBy(...) and insert values like could with pure HTML elements).

Edit - another unsuccessful attempt

I also tried this following this:

user <- "MYUSER"
pw <- "MYPASSWORD@123"

script <- glue::glue(
  "browserstack_executor: {\"action\": \"sendBasicAuth\", \"arguments\": {\"username\": \"{{user}}\", \"password\": \"{{pw}}\", \"timeout\": \"1000\"}}",
  .open = "{{", .close = "}}"
)
dr$executeScript(script)

but I get the error: Dismissed user prompt dialog: This site is asking you to sign in.

Edit 2 - Docker website for MWE

If you want to create a simple web app with basic auth for debugging, you can use the following docker image and command:

file nginx.conf

# see also https://stackoverflow.com/a/67981564/3048453
server {
  listen 80 default_server;
  listen [::]:80 default_server;
  location / {
    default_type text/plain;
    return 200 "Welcome";
  }
  location /auth {
    auth_basic "Restricted";
    auth_basic_user_file "/etc/nginx/.htpasswd";
    proxy_pass http://127.0.0.1:8080;
  }
}
server {
  listen 8080;
  location / {
    default_type text/plain;
    return 200 "AUTHENTICATED";
  }
}

file Dockerfile

FROM nginx:1.23.1
RUN apt-get update && apt-get install -y apache2-utils && mkdir -p etc/nginx && htpasswd -c -B -b /etc/nginx/.htpasswd user password
COPY nginx.conf /etc/nginx/conf.d/default.conf
RUN service nginx restart

Command to start the container: docker build -t nginx-so-tester . && docker run --rm -p 80:80 nginx-so-tester

And then you can visit http://localhost/auth. The credentials are user and password.

Edit 3 - GUI of Remote Docker Instance

While debugging this problem, I setup a docker instance to which I can connect to and look at what is happening on the browser side.

I start a docker selenium instance with docker run --rm -p 4444:4444 -p 7900:7900 --shm-size="2g" selenium/standalone-firefox:103.0 and then connect from RSelenium to it with:

dr <- remoteDriver(
  remoteServerAddr = "localhost",
  port = 4444L
)
dr$open()

at the same time I open my browser to localhost:7900 and provide the password secret (default pw by selenium) and connect to the VNC and see the following:

When I navigate to the problematic URL like so

url <- "http://my.redacted-url.com"
dr$navigate(url)

I see the following in the VNC browser:

I also tried to use dr$sendKeysToAlert(list("user", keys = "TAB", "password", keys = "ENTER")) but get a 401 (ie the authorization didnt work because the credentials where not pasted correctly to the form, I guess).

Answer 1

After some debugging, we were able to change the webserver of the service to allow for the credentials in the URL (I had no idea that there are webservers that do not allow this).

That is, the solution is now this:

library(RSelenium)

# start and connect to the selenium server as shown in the question...
dr <- remotDriver(
  ...,
  # note that we also had to allow insecure certificates due to the setup of an internal network
  # avoid if possible
  extraCapabilities = list("acceptInsecureCerts" = TRUE)
)

user <- "admin"
pw <- "admin"

full_url <- paste0("http://", user, ":", URLencode(pw, reserved = TRUE), "@https://the-internet.herokuapp.com/basic_auth")

dr$navigate(full_url)
dr$screenshot(display = TRUE)