As host-gw use IP routes to subnets via remote machine IPs, it looks like pure L3 network solution.
Therefore, why need direct L2 connectivity between hosts?
host-gw adds route table entries on hosts, so that host know how to traffic container network packets.
This works on L2, because it only concerns hosts, switches and containers. switches does not care IP and route, hosts know containers exists, and how to route to them, containers just send and receive data.
If hosts are at different networks, L3 is introduced, and routers are involved. routers have no idea that containers exists, and any containers packet will be dropped, making communication impossible.
Of course, you can add route table entries on routers, but that is out of control flannel.