After deploying an Azure ML model to a container instance, call to the model fails when using the code provided in the "Consume" section of the endpoint (Python and C#).
I have trained a model in Azure Auto-ML and deployed the model to a container instance.
Now when I am try to use the Python code provided in the Endpoint's "Consume" section I get the following error:
The request failed with status code: 502
Content-Length: 55
Content-Type: text/html; charset=utf-8
Date: Mon, 07 Mar 2022 12:32:07 GMT
Server: nginx/1.14.0 (Ubuntu)
X-Ms-Request-Id: 768c2eb5-10f3-4e8a-9412-3fcfc0f6d648
X-Ms-Run-Function-Failed: True
Connection: close
---------------------------------------------------------------------------
JSONDecodeError Traceback (most recent call last)
<ipython-input-1-6eeff158e915> in <module>
48 # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
49 print(error.info())
---> 50 print(json.loads(error.read().decode("utf8", 'ignore')))
/anaconda/envs/azureml_py36/lib/python3.6/json/init.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
352 parse_int is None and parse_float is None and
353 parse_constant is None and object_pairs_hook is None and not kw):
--> 354 return _default_decoder.decode(s)
355 if cls is None:
356 cls = JSONDecoder
/anaconda/envs/azureml_py36/lib/python3.6/json/decoder.py in decode(self, s, _w)
337
338 """
--> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
340 end = _w(s, end).end()
341 if end != len(s):
/anaconda/envs/azureml_py36/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
355 obj, end = self.scan_once(s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
If I use C# code provided in the Endpoint's "Consume" section I get the following error:
The request failed with status code: BadGateway
Connection: keep-alive
X-Ms-Request-Id: 5c3543cf-29ac-46a3-a9fb-dcb6a0041b08
X-Ms-Run-Function-Failed: True
Date: Mon, 07 Mar 2022 12:38:32 GMT
Server: nginx/1.14.0 (Ubuntu)
'<=' not supported between instances of 'str' and 'int'
The Python code I am using:
import urllib.request
import json
import os
import ssl
def allowSelfSignedHttps(allowed):
# bypass the server certificate verification on client side
if allowed and not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None):
ssl._create_default_https_context = ssl._create_unverified_context
allowSelfSignedHttps(True) # this line is needed if you use self-signed certificate in your scoring service.
data = {
"Inputs": {
"data":
[
{
"SaleDate": "2022-02-08T00:00:00.000Z",
"OfferingGroupId": "0",
"week_of_year": "7",
"month_of_year": "2",
"day_of_week": "1"
},
]
},
"GlobalParameters": {
"quantiles": "0.025,0.975"
}
}
body = str.encode(json.dumps(data))
url = 'http://4a0427c2-30d4-477e-85f5-dfdfdfdfdsfdff623f.uksouth.azurecontainer.io/score'
api_key = '' # Replace this with the API key for the web service
headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}
req = urllib.request.Request(url, body, headers)
try:
response = urllib.request.urlopen(req)
result = response.read()
print(result)
except urllib.error.HTTPError as error:
print("The request failed with status code: " + str(error.code))
# Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
print(error.info())
print(json.loads(error.read().decode("utf8", 'ignore')))
The C# code I have tried:
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace MLModelAPICall
{
class Program
{
static void Main(string[] args)
{
InvokeRequestResponseService().Wait();
}
static async Task InvokeRequestResponseService()
{
var handler = new HttpClientHandler()
{
ClientCertificateOptions = ClientCertificateOption.Manual,
ServerCertificateCustomValidationCallback =
(httpRequestMessage, cert, cetChain, policyErrors) => { return true; }
};
using (var client = new HttpClient(handler))
{
// Request data goes here
var scoreRequest = new
{
Inputs = new Dictionary<string, List<Dictionary<string, string>>>()
{
{
"data",
new List<Dictionary<string, string>>()
{
new Dictionary<string, string>()
{
{
"SaleDate", "2022-02-08T00:00:00.000Z"
},
{
"OfferingGroupId", "0"
},
{
"week_of_year", "7"
},
{
"month_of_year", "2"
},
{
"day_of_week", "1"
}
}
}
}
},
GlobalParameters = new Dictionary<string, string>()
{
{
"quantiles", "0.025,0.975"
}
}
};
const string apiKey = ""; // Replace this with the API key for the web service
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey);
client.BaseAddress = new Uri("http://4a0427c2-30d4-477e-85f5-xxxxxxxxxxxxx.uksouth.azurecontainer.io/score");
// WARNING: The 'await' statement below can result in a deadlock
// if you are calling this code from the UI thread of an ASP.Net application.
// One way to address this would be to call ConfigureAwait(false)
// so that the execution does not attempt to resume on the original context.
// For instance, replace code such as:
// result = await DoSomeTask()
// with the following:
// result = await DoSomeTask().ConfigureAwait(false)
var requestString = JsonConvert.SerializeObject(scoreRequest);
var content = new StringContent(requestString);
content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
HttpResponseMessage response = await client.PostAsync("", content);
if (response.IsSuccessStatusCode)
{
string result = await response.Content.ReadAsStringAsync();
Console.WriteLine("Result: {0}", result);
}
else
{
Console.WriteLine(string.Format("The request failed with status code: {0}", response.StatusCode));
// Print the headers - they include the requert ID and the timestamp,
// which are useful for debugging the failure
Console.WriteLine(response.Headers.ToString());
string responseContent = await response.Content.ReadAsStringAsync();
Console.WriteLine(responseContent);
Console.ReadLine();
}
}
}
}
}
Could you please help me with this issue? I am not sure what do to if Microsoft's provided code is erroring out, don't know what else to do.
After much more digging I found out that the "Consume" scripts provided with the endpoint are wrong (Python and C#) .
When making a call to the endpoint the GlobalParameters expects an integer value, but the provided scripts have wrapped the values in double quotes hence making it a string:
},
"GlobalParameters": {
"quantiles": "0.025,0.975"
}
If you are using Python to consume the model, when making call to the endpoint your GlobalParameters should be define as this:
},
"GlobalParameters": {
"quantiles": [0.025,0.975]
}
wrapped in square brackets
[0.025,0.975]
and not in double quotes "
I have also opened a ticket with microsoft so hopefully they will fix the code provided in the "consume" section of every endpoint