I want to get access to a json page that contains data.
Say, it's there under http://example.com/home/apples.json
It's protected, so a login page is shown.
If I were to visit it manually I first go to http://example.com/ There I get a login menu and after login as say user "test" with password "test" visiting http://example.com/home/apples.json again will show me the json data.
If I do a
curl -u test:test http://example.com/home/apples.json -v
I end up at the login page. (Despite the authentication.)
If I visit http://example.com/ and manually login I will get a cookie after login. Say this cookie is called ltoken and gets the value "dj8f". It will only show up after a successfull login. When I look it up via the browser and copy the cookie data and attach it to the curl command:
curl -b "ltoken="dj8f" http://example.com/home/apples.json -v
it will work.
How can I get the cookie data from after login without doing it manually? Is it possible via bash shell scripting means? If yes how? Alternatively how would it be done in a groovy script?
You have to perform a proper login with curl first. In order to do this, navigate to the login page and have a look at the source code. The login form should look something like this.
<form action="http://example.com/login/target" method="POST">
<input type="text" name="username" />
<input type="password" name="passphrase" />
<input type="submit" value="Log in" />
</form>
(I assume that there are no hidden input fields. Hidden input fields are often used to protect against request forgery, which is basically what we are going to do.)
From this snipped you have to extract the target of the login request (in this example http://example.com/login/target) the names of the HTML input fields (here username and passphrase). To perform the login process, you should send the login information to the target, for example by executing
curl --cookie-jar cookies.txt --form passphrase=test --form username=test http://example.com/login/target
(Please note, that is generally not advisable to type your password on the command line in this way. Your password is probably stored in a command history and could be stolen.)
The --cookie-jar option tells curl to store all cookies in a file. If the login succeeds and the server sets session cookies, curl will save them in this text file.
After a successful login, you should be able to access the json-file, if the request contains the cookies from the cookie-jar. This can be achieved with the -b argument. If -b does not contain the = character, curl will use the contents of the file as cookies.
curl -b cookies.txt http://example.com/home/apples.json -v