HTTP authentication with Apache HTTP Components: force sending of challenge

Question

I need to talk to an obscure webserver which requires authentication. If I don't supply credentials, a login form is displayed. However, if I do supply unsolicited Basic Authentication credentials, I get directly to the desired content.

wget supports this directly:

# this fails and downloads a form:
wget https://weird.egg/data.txt --http-user=me --http-password=shhh

# this works and downloads the document:
wget https://weird.egg/data.txt --http-user=me --http-password=shhh --auth-no-challenge

Now my question: How can I make the download in Java using Apache's HTTP Components?

Here's what I got so far. (There's also a proxy in place, and I use -Y on in wget, and I have a matching https_proxy environment variable.)

import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import java.net.URI;

// ...

DefaultHttpClient hc = new DefaultHttpClient();
hc.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, new HttpHost(proxy_name, proxy_port));

URI uri = new URI("https://weird.egg/data.txt");

hc..getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM, AuthScope.ANY_SCHEME), new UsernamePasswordCredentials("me", "shh"));

hc.execute(new HttpGet(uri)); // etc

However, I only end up with the login form page, not the actual document. I'm suspecting that the DefaultHttpClient isn't sending the credentials unsolicited, in the way that wget does. Is there a way to make the Java program send the credentials?

Answer 1

Never mind. I solved the problem by not trying to use any library authentication methods, but just brute-forcing the Basic Authentication header into the request:

HttpGet get = new HttpGet(uri);

String basic_auth = new String(Base64.encodeBase64((username + ":" + password).getBytes()));
get.addHeader("Authorization", "Basic " + basic_auth);

hc.execute(get); // etc

(This needs the additional import org.apache.commons.codec.binary.Base64;, but in turn we can remove the credential-related imports.)