using OpenTSDB HTTP api in .NET : 400 Bad Request

Question

I'm trying to use .net to put datapoints in OpenTSDB, using the HTTP /api/put API. I've tried with httpclient, webRequest and HttpWebRequest. The outcome is always 400 - bad request: chunked request not supported.

I've tried my payload with an api tester (DHC) and works well. I've tried to send a very small payload (even plain wrong, like "x") but the reply is always the same.

Here's one of my code instances:

    public async static Task  PutAsync(DataPoint dataPoint)
    {
        try
        {
            HttpWebRequest http = (HttpWebRequest)WebRequest.Create("http://127.0.0.1:4242/api/put");
            http.SendChunked = false;
            http.Method = "POST";

            http.ContentType = "application/json";

            Encoding encoder = Encoding.UTF8;
            byte[] data = encoder.GetBytes( dataPoint.ToJson() + Environment.NewLine);
            http.Method = "POST";
            http.ContentType = "application/json; charset=utf-8";
            http.ContentLength = data.Length;
            using (Stream stream = http.GetRequestStream())
            {
                stream.Write(data, 0, data.Length);
                stream.Close();
            }

            WebResponse response = http.GetResponse();

            var streamOutput = response.GetResponseStream();
            StreamReader sr = new StreamReader(streamOutput);
            string content = sr.ReadToEnd();
            Console.WriteLine(content);
        }
        catch   (WebException exc)
        {
            StreamReader reader = new StreamReader(exc.Response.GetResponseStream());
            var content = reader.ReadToEnd();
        }

                    return    ;
    }

where I explicitly set to false the SendChunked property.

note that other requests, like:

 public static async Task<bool> Connect(Uri uri)
        {
            HttpWebRequest http = (HttpWebRequest)WebRequest.Create("http://127.0.0.1:4242/api/version");
            http.SendChunked = false;
            http.Method = "GET";
            // http.Headers.Clear();
            //http.Headers.Add("Content-Type", "application/json");
            http.ContentType = "application/json";
            WebResponse response = http.GetResponse();

            var stream =   response.GetResponseStream();
            StreamReader sr = new StreamReader(stream);
            string content = sr.ReadToEnd();
            Console.WriteLine(content);
            return true;

        }

work flawlessly. I am sure I am doing something really wrong. I'd like to to reimplement HTTP in Sockets from scratch.

Answer 1

I've found a solution I'd like to share here. I've used wireshark to sniff my packets, and I've found that this header is added:

Expect: 100-continue\r\n

(see 8.2.3 of https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html)

This is the culprit. I've read the post http://haacked.com/archive/2004/05/15/http-web-request-expect-100-continue.aspx/ by Phil Haack, and found that HttpWebRequest puts that header by default, unless you tell it to stop. In this article I've found that using ServicePointManager I can do just this.

Putting the following code on top of my method, when declaring the http object, makes it work very well, and solves my issue:

            var uri = new Uri("http://127.0.0.1:4242/api/put");
            var spm = ServicePointManager.FindServicePoint(uri);
            spm.Expect100Continue = false;
            HttpWebRequest http = (HttpWebRequest)WebRequest.Create(uri); 
            http.SendChunked = false;