Search code examples
amazon-web-servicesdnsamazon-cloudfront

Using CloudFront to split traffic between origin servers – is this configuration right?

I'm replacing a legacy website with a new one, in stages. From a user perspective, all existing URLs must remain the same, but certain paths should serve new pages.

I have two origin servers: the legacy one (www.mysite.com) and the new, EC2 one (www.ec2-loadbalancer.com) – obviously mock URLs for privacy reasons.

I've created an AWS CloudFront distribution with a CNAME setting of www.mysite.com. Within this distribution I've created two origins domains:

  • www.mysite.com
  • www.ec2-loadbalancer.com

Within CloudFront I've configured some behaviours so that paths like /foo are sent to my EC2 load balancer origin, and all other paths (eg. default) are sent to www.mysite.com.

From a DNS perspective, I've added a CNAME record of www.mysite.com which points to my Cloudfront host domain (eg. foo.cloudfront.net.). The A record for this domain points to the IP of the legacy server.

I launched all of this today and it seems to have worked, but I'm seeing intermittent 403 errors on the site, and two hours after making the change (there was no "www" CNAME before, so TTL shouldn't make a difference), some browsers are still serving the site from the original IP (rather than the CloudFront one).

Have I configured this properly? I couldn't work out how to do this via the A record – that points to the IP of the legacy server, and CloudFront doesn't allow me to enter an IP address as an origin. Should I have pointed the www CNAME at the IP address, made the A record point at CloudFront instead? I'm a bit lost here.

On the other hand, it could all be a propagation thing, but I'm wary having seen 40x errors hours after making the change.

Solution

  • It's a shame you can't share the domains with us to help debugging, but at the very least, dig is your friend. For instance:

    $ dig membership.theguardian.com
...
;; ANSWER SECTION:
membership.theguardian.com. 367 IN  CNAME   i.global-ssl.fastly.net.
i.global-ssl.fastly.net. 12 IN  A   151.101.128.67
i.global-ssl.fastly.net. 12 IN  A   151.101.64.67
i.global-ssl.fastly.net. 12 IN  A   151.101.0.67
i.global-ssl.fastly.net. 12 IN  A   151.101.192.67

    ...tells you that membership.theguardian.com is pointing to Fastly by CNAME. You can check with alternate DNS servers, like Google's DNS on 8.8.8.8, by doing this:

    $ dig @8.8.8.8 membership.theguardian.com

    ...so you can see how other people are resolving your domains.

    From a DNS perspective, I've added a CNAME record of www.mysite.com which points to my Cloudfront host domain (eg. foo.cloudfront.net.). The A record for this domain points to the IP of the legacy server.

    I'm not a DNS expert, so it's possible I'm not understanding you here, but this sound like it introduces ambiguity? To me this sounds like you have two different records for the same www.mysite.com domain, one of which points to CloudFront, and the other to the IP of your legacy server. Depending on how that gets resolved a browser could be sent to one or the other?!

    • www.mysite.com should point only to CloudFront. I would personally just use a CNAME for this.
    • You should have unambiguous addresses for both your legacy server and your EC2 Elastic Load Balancer - I would personally give them their own clear domain names, to avoid confusion (eg. legacy.mysite.com & beta.mysite.com) - and in CloudFront refer to only to those clear names when you're directing traffic (eg. passing traffic on to www.mysite.com as a way of going to the legacy server would be confusing).

    enter image description here

    Good luck!