I'm using an inputFile button to upload some files...
The button renders on my page as a grey 'Choose File To Upload' button. Can I add my own custom icon/image in place of the button?
Yes you can do that. Although, it is not a straight forward solution.
The solution is to :
You may find the solution described in detail with code on my blog post --> https://sfdc-gyaan.rhcloud.com/2014/05/renamingoverriding-choose-filebrowse-button-in-salesforce/
I hope this helps. Let me know if you face any issues implementing it.