Search code examples
linuxbashcookiescurlhttp-authentication

CURL to access a page that requires a login from a different page


I have 2 pages: xyz.example/a and xyz.example/b. I can only access xyz.example/b if and only if I login to xyz.example/a first. If accessing xyz.example/b without going through the other, I simply get access denied (no redirect to login) via the browser. Once I login at xyz.example/a, I can access the other.

My problem is doing this using the curl command. I can login successfully to xyz.example/a using curl, but then try xyz.example/b and I get access denied.

I use the following:

curl --user user:pass https://xyz.example/a  #works ok
curl https://xyz.example/b #doesn't work

I've tried using the second line with & without the user/password part and still doesn't work. Both pages uses the same CA, so that's not a problem.


Solution

  • The web site likely uses cookies to store your session information. When you run

    curl --user user:pass https://xyz.example/a  #works ok
    curl https://xyz.example/b #doesn't work
    

    curl is run twice, in two separate sessions. Thus when the second command runs, the cookies set by the 1st command are not available; it's just as if you logged in to page a in one browser session, and tried to access page b in a different one.

    What you need to do is save the cookies created by the first command:

    curl --user user:pass --cookie-jar ./somefile https://xyz.example/a
    

    and then read them back in when running the second:

    curl --cookie ./somefile https://xyz.example/b
    

    Alternatively you can try downloading both files in the same command, which I think will use the same cookies.