Website login and keep session cookies

Question

I am trying to scrape some content form a website but you must be logged in order to view specific content. I want make a login using user id & password and keep session cookies on: m.amway.com i tried using Jsoup.... however after using the code below i realize that Jsoup cannot read javascript which is what the website is based on.... Does anyone have a method i could use to login, keep session cookie, and scrape content, using something other than Jsoup? Thanks in advance.

public String Jlogin(String User, String Pass) throws Exception{

    String title = "didnt work";

        Response logRes = Jsoup.connect(AmwayURL)
                .data("userid", User)
                .data("userpswd", Pass)
                .method(Method.POST)
                .execute();
        // get all cookies
        Map<String, String> cookies = logRes.cookies();
        Document doc1 = logRes.parse();
        String sessionId = logRes.cookie("JSESSIONID");
        Document doc2 = Jsoup
                .connect("https://m.amway.com/business/volume/pvbv/inquiry.ashx")
                .cookie("jsessionid", sessionId).get();
        System.out.println(doc2);

        title = doc2.toString() + "................." + sessionId;

    return title;
}

Answer 1

You can use a much larger API called HttpClient.

has the following classes: - HttpGet - HttpPost - HttpEntity - HttpResponse

HttpResponse reads Javascript from any page, as follows: EntityUtils.toString(HttpResponse.getEntity());

for more details on how to use the API, check this link (Extremly helps): http://www.codeblues.in/blog/?p=5