C# Create Deedle DataFrame from JSON response

Question

I'm having a bit of trouble loading a JSON response from this request into a Deedle DataFrame:https://sampleserver6.arcgisonline.com/arcgis/rest/services/Earthquakes_Since1970/FeatureServer/0/query?where=OBJECTID%3C10&returnGeometry=false&f=json

In the JSON, what I'm interested are the features. More specifically, for each feature there are attributes - I want essentially just a collection of those attributes to load into a DataFrame. In this particular case, there is only one attribute "name" so my expectation is that the resulting DataFrame would have a column "name" with the values shown.

I've tried using json2csharp and creating my own class, but the result either doesn't have the column header/values or the values are missing. I'm not really sure what I'm doing wrong or if I'm even approaching this the right way. My understanding from the Deedle documentation is that it should be possible to create a DataFrame from a collection of objects: https://bluemountaincapital.github.io/Deedle/csharpframe.html#Creating-and-loading-data-frames. Certainly, using the Enumerable example listed on the page works as expected.

Here is the pertinent section of my code:

        string url = "https://sampleserver6.arcgisonline.com/arcgis/rest/services/Earthquakes_Since1970/FeatureServer/0/query?";
        WebClient wc = new WebClient();

        wc.QueryString.Add("where", "OBJECTID<10");
        wc.QueryString.Add("returnGeometry", "false");
        wc.QueryString.Add("f", "json");
        var data = wc.UploadValues(url, "POST", wc.QueryString);

        var responseString = UnicodeEncoding.UTF8.GetString(data);
        JObject o = JObject.Parse(responseString);
        dynamic x = JsonConvert.DeserializeObject(responseString);
        var testObjList = new List<dynamic>();
        foreach (dynamic element in x.features)
            {

                testObjList.Add(new myClass { name = element.attributes.name});

                Console.WriteLine($"{element.attributes.name}");
            }


        var dfObjects = Frame.FromRecords(testObjList);
        dfObjects.Print();
        var df = Frame.FromRecords(test);
        df.Print(); // No headers or values shown

where myClass is just this:

   public class myClass{

        public string name { get; set; }

    }

Any help/pointers would be much appreciated!

Answer 1

The Frame.FromRecords operation relies on static type information to figure out what properties the class has. In your case, you define the list of objects as List<dynamic> - this is compiled as Object and so Deedle does not see any members.

To fix this, all you need to do is to define the type as a list of myClass objects:

var testObjList = new List<myClass>();

A more compact approach using an anonymous type would work too:

var testObjList =       
  ((IEnumerable<dynamic>)x.features).Select(element => 
    new { name = element.attributes.name });
var dfObjects = Frame.FromRecords(testObjList);