Search code examples
jsonapama

Mapping JSON to Apama Events

I am trying to use the Mapper codec in my connectivity chain to convert a JSON object that looks like this:

{"test2":[
    ["column1","column2","column3"],
    ["16091", "449", "05302018"],
    ["16092", "705", "05302018"]
]}

to an EPL type. To me it looks like a sequence of sequences, so I used

event test1 {
    sequence<string> values;
}

event test2 {
    sequence<test1> tests;
}

But this gives me the error 

Unable to parse event test.1: Incorrect type in get (you asked for map but its' actually list)

Any ideas how I should be using the Mapper codec to this end?

Solution

  • Unless explicitly remapped, that won't quite work. You have to consider the entire structure of the document from top to bottom. It's not a sequence of strings - it's a JSON object/dictionary at top-level, with a value that is a sequence of sequences of string.

    A JSON object/dictionary can map to an event type based on field names. So as Matt's answer said, a JSON document like yours would need an event type like

    event SomeEventType {
    sequence<sequence<string > > test2;
}

    If it's not appropriate to create an event type that exactly corresponds to the JSON document's structure, then you'll need to use the mapping codec to rearrange the fields in the JSON document to match the fields and sub-fields in an event type. Or possibly a custom codec; I think Matt's right that the mapper can't do exactly what you want.

    Further, because JSON documents are type-less at the top-level, you'll need to make sure that the event type is defined somehow. There are multiple ways of doing that.

    (1) If this particular connectivity will only send you events of one type, you can use the 'defaultEventType' configuration option of the apama.eventMap host plug-in at the top of your chain e.g.

    apama.eventMap:
    defaultEventMap: SomeEventType

    (2) If it depends on the structure of the document, you'll need to use the classifier codec. That can take a message going towards the correlator, and assign it an event type based on the content of fields (or simply their presence). You can learn about it in the documentation.

    (3) The transport will sometimes define it on messages being sent towards the correlator. For example, in the case of the Universal Messaging transport, then the 'tag' of the UM event will be used as the type. This may or may not be appropriate.

    If you do end up doing anything non-trivial with the classifier or mapper, I'd strongly recommend use of the 'diagnostic codec' to help in developing the classifier or mapper rules. This is a codec you can put anywhere in the chain of codecs that will log every event going through it, so you can see how your rules are operating by seeing what happens before and after classification/mapping. You can read about it in the documentation, but it's usually as simple as putting '- diagnosticCodec' somewhere in your chain. I've found it absolutely invaluable when debugging connectivity chains.