Search code examples
javacurlhttpclienthttp-getcookiestore

Java HTTP POST and GET converted from CURL


Hey all I am trying my best to convert this working CURL script to Java:

@ECHO off
set COOKIES=.\cookies.txt
set USER=myUN
set PASSWORD="myPW"
set HOST=https://zzzzzzz.zzzzz.zz.zz:9443

cls

ECHO "Get the session cookie-------------------------------------"
    set out1 = curl -k -c %cookies% "%host%/qm/authenticated/identity" > nul
ECHO "-----------------------------------------------------------"

ECHO "Use the session cookie to logon----------------------------"
    curl -k -L -b %COOKIES% -c %COOKIES% -d j_username=%USER% -d j_password=%PASSWORD% %host%/qm/authenticated/j_security_check > nul
ECHO "-----------------------------------------------------------"

ECHO "Use the cookie to get the catalog--------------------------"
    curl -k -L -b %COOKIES% -H "Accept: application/xml" %host%/qm/process/project-areas/_zzzzzzzzzzzzzzzzzzzz/members
ECHO "-----------------------------------------------------------"

where:

               -k: Allow insecure server connections when using SSL
    -c <filename>: Write cookies to <filename> after operation
               -L: Follow redirects
        -b <data>: Send cookies from string/file
-H <header/@file>: Pass custom header(s) to server
        -d <data>: HTTP POST data

I've searched around for java code that includes an example of creating a cookie but I have yet to find something that would help me with my code above.

The closest I could find and modify the POST call is this:

CookieStore cookieStore = new BasicCookieStore();
String USER     = "myUN";
String PASSWORD = "myPW";
String HOST     = "https://zzzzzzz.zzzzz.zz.zz:9443";

// CookieSpecs.STANDARD is the RFC 6265 compliant policy
RequestConfig requestConfig = RequestConfig
                              .custom()
                              .setCookieSpec(CookieSpecs.STANDARD)
                              .build();

// automatically follow redirects
CloseableHttpClient client = HttpClients
                            .custom()
                            .setRedirectStrategy(new LaxRedirectStrategy())
                            .setDefaultRequestConfig(requestConfig)
                            .setDefaultCookieStore(cookieStore)
                            .build();
HttpPost postIT                  = new HttpPost(HOST + "/qm/authenticated/identity");
List<NameValuePair> urlParams    = new ArrayList<>();

urlParams.add(new BasicNameValuePair("j_username", USER));
urlParams.add(new BasicNameValuePair("j_password", PASSWORD));
postIT.setEntity(new UrlEncodedFormEntity(urlParams));

HttpResponse mmmCookie = client.execute(postIT);
// ... we have our cookie!

For postIT the value is:

POST https://zzzzz.zzz.zzzz.zzzz:9443/qm/authenticated/identity HTTP/1.1

However, it has an error of:

error: null

Not sure why the error is null since postIT has data? So not only am I not able to run the modified code I did, I'm still wondering how I go about calling a GET command using the cookie even if the code above worked.

So, in a nutshell:

  • Fix httpPost error.
  • How to send cookie on other GET methods.

Help would be great!

update for VGR:

public static void main(String[] args) throws IOException {
    Path currentRelativePath = Paths.get("").toAbsolutePath();
    PermissiveTrustManager blah = new PermissiveTrustManager();

    blah.readMembers("https://zzzzzz.zzzz.zz.zzz:9443", "zzzzzz", "zzzzzzz", currentRelativePath);
}

The error is on this line:

check(tm -> tm.checkServerTrusted(certChain, authType, socket), socket);

certChain, authType and socket all have data in them.


Solution

  • Since we don’t know what’s causing your cryptic error, I would suggest abandoning the third party library, and using the java.net package:

    public void readMembers(String schemeAndAuthority,
                            String username,
                            String password,
                            Path membersFileToWrite)
    throws IOException {
    
        URI baseURI = URI.create(schemeAndAuthority);
    
        CookieHandler oldCookieHandler = CookieHandler.getDefault();
        boolean oldFollowRedirects = HttpURLConnection.getFollowRedirects();
    
        CookieHandler.setDefault(new CookieManager());
        HttpURLConnection.setFollowRedirects(true);
    
        try {
            HttpURLConnection connection;
    
            URI authURI = baseURI.resolve("/qm/authenticated/identity");
            connection = (HttpURLConnection) authURI.toURL().openConnection();
            connection.getResponseCode();
    
            URI securityURI = baseURI.resolve(
                "/qm/authenticated/j_security_check");
            String postData = 
                "j_username=" + URLEncoder.encode(username, "UTF-8") + "&" +
                "j_password=" + URLEncoder.encode(password, "UTF-8");
            connection = (HttpURLConnection)
                securityURI.toURL().openConnection();
            connection.setRequestMethod("POST");
            connection.setRequestProperty("Content-Type",
                "application/x-www-form-urlencoded");
            connection.setDoOutput(true);
            try (OutputStream postBody = connection.getOutputStream()) {
                postBody.write(postData.getBytes(StandardCharsets.UTF_8));
            }
            connection.getResponseCode();
    
            URI catalogURI = baseURI.resolve(
                "/qm/process/project-areas/_zzzzzzzzzzzzzzzzzzzz/members");
            connection = (HttpURLConnection)
                catalogURI.toURL().openConnection();
            connection.setRequestProperty("Accept", "application/xml");
            try (InputStream responseBody = connection.getInputStream()) {
                Files.copy(responseBody, membersFileToWrite);
            }
        } finally {
            CookieHandler.setDefault(oldCookieHandler);
            HttpURLConnection.setFollowRedirects(oldFollowRedirects);
        }
    }
    

    However, the above code does not provide the equivalent of curl’s -k option. For that, we have to create a custom SSLContext, initialized with a TrustManager that allows all certificates:

    static class PermissiveTrustManager
    extends X509ExtendedTrustManager {
        private final X509ExtendedTrustManager[] realTrustManagers;
    
        private interface Checker {
            void checkWith(X509ExtendedTrustManager realTrustManager)
            throws CertificateException;
        }
    
        PermissiveTrustManager() {
            TrustManagerFactory factory;
            try {
                factory = TrustManagerFactory.getInstance(
                    TrustManagerFactory.getDefaultAlgorithm());
                factory.init((KeyStore) null);
            } catch (GeneralSecurityException e) {
                // We should not be able to get here.
                throw new RuntimeException(e);
            }
    
            TrustManager[] allTrustManagers = factory.getTrustManagers();
            realTrustManagers = Arrays.stream(allTrustManagers)
                .filter(tm -> tm instanceof X509ExtendedTrustManager)
                .map(tm -> (X509ExtendedTrustManager) tm)
                .toArray(X509ExtendedTrustManager[]::new);
    
        }
    
        private void check(Checker checker) {
            try {
                for (X509ExtendedTrustManager realTrustManager : realTrustManagers) {
                    checker.checkWith(realTrustManager);
                }
            } catch (CertificateException e) {
                System.err.println("Ignoring invalid certificate");
                e.printStackTrace();
            }
        }
    
        private void check(Checker checker,
                           Socket socket) {
            try {
                for (X509ExtendedTrustManager realTrustManager : realTrustManagers) {
                    checker.checkWith(realTrustManager);
                }
            } catch (CertificateException e) {
                System.err.println("Ignoring invalid certificate for " + 
                    socket.getRemoteSocketAddress());
                e.printStackTrace();
            }
        }
    
        @Override
        public void checkClientTrusted(X509Certificate[] certChain,
                                       String authType,
                                       Socket socket) {
            check(tm -> tm.checkClientTrusted(certChain, authType, socket), socket);
        }
    
        @Override
        public void checkClientTrusted(X509Certificate[] certChain,
                                       String authType,
                                       SSLEngine engine) {
            check(tm -> tm.checkClientTrusted(certChain, authType, engine));
        }
    
        @Override
        public void checkServerTrusted(X509Certificate[] certChain,
                                       String authType,
                                       Socket socket) {
            check(tm -> tm.checkServerTrusted(certChain, authType, socket), socket);
        }
    
        @Override
        public void checkServerTrusted(X509Certificate[] certChain,
                                       String authType,
                                       SSLEngine engine) {
            check(tm -> tm.checkServerTrusted(certChain, authType, engine));
        }
    
        @Override
        public void checkClientTrusted(X509Certificate[] certChain,
                                       String authType) {
            check(tm -> tm.checkClientTrusted(certChain, authType));
        }
    
        @Override
        public void checkServerTrusted(X509Certificate[] certChain,
                                       String authType) {
            check(tm -> tm.checkServerTrusted(certChain, authType));
        }
    
        @Override
        public X509Certificate[] getAcceptedIssuers() {
            return new X509Certificate[0];
        }
    }
    
    private static SSLContext createPermissiveSSLContext()
    throws IOException {
    
        TrustManager[] trustManagers = { new PermissiveTrustManager() };
    
        try {
            SSLContext context = SSLContext.getInstance("TLS");
            context.init(null, trustManagers, null);
            return context;
        } catch (GeneralSecurityException e) {
            throw new IOException(e);
        }
    }
    
    public void readMembers(String schemeAndAuthority,
                            String username,
                            String password,
                            Path membersFileToWrite)
    throws IOException {
    
        URI baseURI = URI.create(schemeAndAuthority);
    
        CookieHandler oldCookieHandler = CookieHandler.getDefault();
        boolean oldFollowRedirects = HttpURLConnection.getFollowRedirects();
        SSLContext oldSSLContext = null;
        try {
            oldSSLContext = SSLContext.getDefault();
        } catch (GeneralSecurityException e) {
            e.printStackTrace();
        }
    
        CookieHandler.setDefault(new CookieManager());
        HttpURLConnection.setFollowRedirects(true);
        SSLContext.setDefault(createPermissiveSSLContext());
    
        try {
            HttpURLConnection connection;
    
            URI authURI = baseURI.resolve("/qm/authenticated/identity");
            connection = (HttpURLConnection) authURI.toURL().openConnection();
            connection.getResponseCode();
    
            URI securityURI = baseURI.resolve(
                "/qm/authenticated/j_security_check");
            String postData = 
                "j_username=" + URLEncoder.encode(username, "UTF-8") + "&" +
                "j_password=" + URLEncoder.encode(password, "UTF-8");
            connection = (HttpURLConnection)
                securityURI.toURL().openConnection();
            connection.setRequestMethod("POST");
            connection.setRequestProperty("Content-Type",
                "application/x-www-form-urlencoded");
            connection.setDoOutput(true);
            try (OutputStream postBody = connection.getOutputStream()) {
                postBody.write(postData.getBytes(StandardCharsets.UTF_8));
            }
            connection.getResponseCode();
    
            URI catalogURI = baseURI.resolve(
                "/qm/process/project-areas/_zzzzzzzzzzzzzzzzzzzz/members");
            connection = (HttpURLConnection)
                catalogURI.toURL().openConnection();
            connection.setRequestProperty("Accept", "application/xml");
            try (InputStream responseBody = connection.getInputStream()) {
                Files.copy(responseBody, membersFileToWrite);
            }
        } finally {
            CookieHandler.setDefault(oldCookieHandler);
            HttpURLConnection.setFollowRedirects(oldFollowRedirects);
            if (oldSSLContext != null) {
                SSLContext.setDefault(oldSSLContext);
            }
        }
    }
    

    Obviously, I have no way to test this.

    You could call the method like this:

    new CatalogRetriever().readMembers(
        "https://zzzzzzz.zzzzz.zz.zz:9443", "myUN", "myPW",
        Paths.get("members"));
    

    As described in the URI docs, the scheme is the http: or https: part. The authority is // followed by a hostname/port (and optional user/password, depending on the protocol).