I've been wanting to setup a linux VirtualBox instance and use iptables to route traffic from other VMs through it and on to an external SOCKS proxy server. However, I have recently read that transparent proxies like this break HTTPS transmissions and that this is part of the design – to prevent man-in-the-middle attacks. If I want this to work, I'll need to configure the proxy inside the browsers of the VM clients for port 443. Are there any other options I could explore to achieve my goal?
To access a SOCKS server each TCP connection must be prefixed with the necessary SOCKS header. That means, that a simple redirect is not possible. You need instead a protocol converter like redsocks or transocks (never used these, but from the description they do what you need).
Apart from that it is not a problem to simply redirect HTTPS traffic or use these protocol converters, as long as you don't change the SSL stream itself. The only problem is if transparent proxies try to intercept and re-route this traffic to other sites (like redirecting to a capture portal) or try to decrypt the connection in order to analyse it (like in firewalls). These kind of interceptions will be noticed from the browser, because either the name in the certificate does not match the target name and/or the issuer of the certificate is not trusted.