Can anyone guide me to where I can download that latest SolrJ release source files?
Solrj is bundled with Solr, so if you download the Solr Source package you will find the Solrj source code included in it.
https://archive.apache.org/dist/lucene/solr/3.2.0/apache-solr-3.2.0-src.tgz
It's located in the directory
solr/src/sorlj