I'm trying to convert JSON I get from a website to a deedle dataframe, expanding the JSON entries to separate columns of the dataframe. I found this discussion, but I can't make the proposed solution work for me. As I'm new to both JSON and deedle, I might be making a silly mistake. I'm trying the following (mostly copied from the cited discussion):
let rec expander key value =
seq {
match value with
| JsonValue.String (s) -> yield key,typeof<string>,box s
| JsonValue.Boolean (b) -> yield key,typeof<bool>,box b
| JsonValue.Float (f) -> yield key,typeof<float>,box f
| JsonValue.Null (_) -> yield key,typeof<obj>,box ()
| JsonValue.Number (n) -> yield key,typeof<decimal>,box n
| JsonValue.Record (r) -> yield! r |> Seq.collect ((<||)expander)
| JsonValue.Array (a) ->
yield! a
|> Seq.collect (expander "arrayItem")
}
Frame.CustomExpanders.Add(typeof<JsonDocument>,
fun o -> (o :?> JsonDocument).JsonValue |> expander "root")
Frame.CustomExpanders.Add(typeof<JsonValue>,
fun o -> o :?> JsonValue |> expander "root")
let info =
JsonValue.Parse(""" { "name": "Tomas", "born": 1985 } """)
let df =
[ series ["It" => info] ]
|> Frame.ofRowsOrdinal
let dfexpanded = Frame.expandAllCols 2 df
This gives me something I don't know how to interpret, but not the desired result:
It.properties It.Tag It.IsString It.IsNumber It.IsFloat It.IsRecord It.IsArray It.IsBoolean It.IsNull It._Print
0 -> System.Tuple`2[System.String,FSharp.Data.JsonValue][] 3 False False False True False False False { "name": "Tomas", "born": 1985 }
I'm thankful for any input!
The problem seems to be that the type of It
in the data frame is not JsonValue
, but one of the sub-classes generated by the compiler to represent individual cases of the discriminated union - in this particular case, a nested type named JsonValue+Record
.
Deedle looks for the exact type match (and does not try to find expander for a base class), so a workaround is to register expander for each nested class:
for t in typeof<JsonValue>.GetNestedTypes() do
Frame.CustomExpanders.Add(t, fun o -> o :?> JsonValue |> expander "root")
After running this, your code gives the expected result:
val dfexpanded : Frame<int,string> =
It.name It.born
0 -> Tomas 1985