HTTP response code different every time i run the script\python 2.7

Question

I wrote simple script to check url's affected by error 404.

When I run the script on url with 404 it indeed returns me correct response however sometimes it returns a different one(503) , I don't understand why this is happening ,I've tested that on many other URL's and the results are never consistent. Would someone be that kind and explain why my results keep changing even though the actual url status doesn't ??

Thanks in advance for assistance

I've tried to use the following modules:

urllib,urllib2, requets

All of them produced the same inconsistent results.

Here are the scripts:

I

import requests

for url in ['https://www.amazon.es/gp/product/B00QTVL0T4']:
    response = requests.get(url)
    response.status_code
    print(response)

Response [404] and Response [503] (incorrect)

II

import urllib

result=''

#***** paste url into square bracket ****#

for url in ["https://www.amazon.es/gp/product/B003ODEJZ2",'https://www.amazon.fr/gp/product/B01H801C9C']:
    a=urllib.urlopen(url)
    e=a.getcode()
    if e==404:
        result+=(url+" Error_404 "+"\n")
    else:
        result+=(url+" Link_OK "+"\n")
print result

III

import urllib2
for url in ['https://www.amazon.es/gp/product/B003ODEJZ2','https://www.amazon.fr/gp/product/B01H801C9C','https://www.amazon.de/dp/B00B8PRE1Y']:
    try:
        connection = urllib2.urlopen(url)
    except urllib2.HTTPError, e:
         if e.getcode()==404:
             print (url+" Error_404")
         else:
             print(url+" Link_Ok")

Answer 1

503's Reason is typically "Service Unavailable" which means that whatever is handling your request cannot find a backing service to supply a response. This may be returned by a physical or software-provided load balancer (F5s, HAProxy, etc.), CDN (Fastly, Cloudflare, etc.), or something like Apache or Nginx that are running but do not have an appropriate application to contact (for whatever reason).

Since the URLs in your example are Amazon URLs, it's likely that the CDN is returning the 503. CDNs (like Fastly) use Varnish and Varnish will return a 503 in the following cases:

• The backend server returned a 503 so Varnish forwards it along
• The backend server returned a response which was not parseable

Some other times a CDN might return a 503:

• The CDN timed out reading from the origin
• The CDN timed out connecting to the origin
• The CDN cannot read/write to the origin
• The origin server refused the connection from the CDN
• The CDN cannot find a route to the origin
• The origin has misconfigured TLS

(And so so so many other problems)

So really, it's not quite possible for us to tell you why you're getting a 503. You need to introspect the response a bit more and look up documentation surrounding that.