Search code examples
javac++jsonlarge-filesjsoniq

Large json file transformation via code or not


after trying to work with mongoDB , and finding out that it dosn't work with documents that are larger the 16 mega .
i need to find way to load large complex json file ( i guess to memory ) .
and to transform it to my need via ( i guess ) Query processor like jsoniq , but i open to other solutions .
the key here that i dont what to change the master json . and i don't care which programming language
to make it ,
but i just want to find method to do it right and fast.


Solution

  • I think the Jackson library can help you. Its official website states the following features:

    Jackson is a:

    1) Streaming (reading, writing)

    2) FAST (measured to be faster than any other Java json parser and data binder)

    3) Powerful (full data binding for common JDK classes as well as any Java bean class, Collection, Map or Enum)

    4) Zero-dependency (does not rely on other packages beyond JDK)

    5)Open Source (LGPL or AL)

    6) Fully conformant

    7) Extremely configurable

    JSON processor (JSON parser + JSON generator) written in Java. Beyond basic JSON reading/writing (parsing, generating), it also offers full node-based Tree Model, as well as full OJM (Object/Json Mapper) data binding functionality.