I need to be able to retrieve an S3 object residing in Australia S3 Storage from London EC2 and minimise latency.
On the London EC2 instance, I set the RegionEndpoint on the S3Client to ApSouthEast2 and acquire it.
The issue I am facing is that the it takes far too long to retrieve even small files (10kb file - 1.2s+). I have Peering setup on the server also so I can ping EC2 instances in ApSouthEast2 from London and my round trips are all 0ms.
However, when retrieving data from S3 it is a different story. How can I allow the s3 bucket to be available "internally" (vpc endpoints and via existing peering??) as I am assuming traversal over WAN is creating this huge lag.
It must be stressed that any changes I make will still require the S3 bucket to be available in its current state as this is a production system.
First, it is impossible to have 0ms latency between Europe and Australia. Something to do with the speed of light. See: Ping time between London and Sydney - WonderNetwork
I would recommend you try to connect to S3 directly (rather than via VPC Peering) and see whether the speed is any better.
Then, try connecting via Amazon S3 Transfer Acceleration and see if it is any better. Connections are routed via the AWS network.
Worst case, you could always replicate the bucket to the London region so that objects are copied soon after they are created. You can then access the objects locally.