Search code examples
javahtmlunitsocks

How can I use SOCKS with HtmlUnit?

Is it possible to use HtmlUnit through SOCKS proxy? Could anyone please provide a code sample?

====

So I've dug through webclient sources, here's the best way I can think of:

  1. Subclass MultiThreadedHttpConnectionManager so that it allows setting SOCKS info and if it is set, before returning a Connection, sets SOCKS parameters

  2. Subclass WebConnection - rewrite createHttpClient so that it uses a manager from step 1 and add a method to get that manager directly or http client at first (it is protected now - so bad...)

  3. To use 1) create a WebClient instance 2) Create subclassed WebConnection 3) Set it to be used by WebClient 4) Access connection's manager and use it's methods to use socks

Solution

  • HtmlUnit uses HttpClient as the underlying connection library, I investigated this a little, but:

    1- Couldn't find a way to configure HttpClient (except by the generic Java Socks mechanism defined in http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html)
2- Do not have access to a public Socks Proxy to test against