I have assigned an egress only internet gateway to my private subnet. Now I can connect with IPV6 websites but not with IPV4 addresses.
Do I need NAT gateways to access IPV4 address from my EC2 machine? (Only outgoing traffic)
You need either a NAT Gateway, or simply an Internet Gateway. But if you are only interested in outgoing, then a NAT Gateway is the way to go.
Egress only internet gateways are for IPv6 only. As mentioned in the documentation for them.