Is there a way to serialize a heterogenous vector with nlohmann_json lib?

Question

Hi Stack Overflow Community !

I am working on a project that heavily uses the interesting nlohmann_json library and it appears that I need to add an inheritance link on a specific class, which objects are serialized at one moment.

I tried different advice found on the github Issues page of the library, but can't make it work.

Here is an dummy code I tried :

#include <nlohmann/json.hpp>

#include <iostream>
#include <memory>
#include <vector>

using json = nlohmann::json;

namespace nlohmann {
    template <typename T>
    struct adl_serializer<std::unique_ptr<T>> {
        static void to_json(json& j, const std::unique_ptr<T>& opt) {
            if (opt) {
                j = *opt.get();
            } else {
                j = nullptr;
            }
        }
    };
}

class Base {
    public:
        Base() = default;
        virtual ~Base() = default;
        virtual void foo() const { std::cout << "Base::foo()" << std::endl; }
};

class Obj : public Base
{
    public:
        Obj(int i) : _i(i) {}
        void foo() const override { std::cout << "Obj::foo()" << std::endl; }
        int _i = 0;
        friend std::ostream& operator<<(std::ostream& os, const Obj& o);
};

std::ostream& operator<<(std::ostream& os, const Base& o)
{
    os << "Base{} ";
    return os;
}

std::ostream& operator<<(std::ostream& os, const Obj& o)
{
    os << "Obj{"<< o._i <<"} ";
    return os;
}

void to_json(json& j, const Base& b)
{
    std::cout << "called to_json for Base" << std::endl;
}

void to_json(json& j, const Obj& o)
{
    std::cout << "called to_json for Obj" << std::endl;
}

int main()
{
    std::vector<std::unique_ptr<Base>> v;
    v.push_back(std::make_unique<Base>());
    v.push_back(std::make_unique<Obj>(5));
    v.push_back(std::make_unique<Base>());
    v.push_back(std::make_unique<Obj>(10));

    std::cout << v.size() << std::endl;

    json j = v;
}
// Results in :
// Program returned: 0
// 4
// called to_json for Base
// called to_json for Base
// called to_json for Base
// called to_json for Base

(https://gcc.godbolt.org/z/dc8h8f)

I understand that the adl_serializer only get the type Base when called, but I don't see how to make him aware of the type Obj as well...

Does anyone see what I am missing here ?

Thanks in advance for your advice and help !

Answer 1

nlohmann.json does not include polymorphic serializing, but you can implement it yourself in a specialized adl_serializer. Here we're storing and checking an additional _type JSON field, used as a key to map to pairs of type-erased from/to functions for each derived type.

namespace PolymorphicJsonSerializer_impl {
    template <class Base>
    struct Serializer {
        void (*to_json)(json &j, Base const &o);
        void (*from_json)(json const &j, Base &o);
    };

    template <class Base, class Derived>
    Serializer<Base> serializerFor() {
        return {
            [](json &j, Base const &o) {
                return to_json(j, static_cast<Derived const &>(o));
            },
            [](json const &j, Base &o) {
                return from_json(j, static_cast<Derived &>(o));
            }
        };
    }
}

template <class Base>
struct PolymorphicJsonSerializer {

    // Maps typeid(x).name() to the from/to serialization functions
    static inline std::unordered_map<
        char const *,
        PolymorphicJsonSerializer_impl::Serializer<Base>
    > _serializers;

    template <class... Derived>
    static void register_types() {
        (_serializers.emplace(
            typeid(Derived).name(),
            PolymorphicJsonSerializer_impl::serializerFor<Base, Derived>()
        ), ...);
    }

    static void to_json(json &j, Base const &o) {
        char const *typeName = typeid(o).name();
        _serializers.at(typeName).to_json(j, o);
        j["_type"] = typeName;
    }

    static void from_json(json const &j, Base &o) {
        _serializers.at(j.at("_type").get<std::string>().c_str()).from_json(j, o);
    }
};

Usage:

// Register the polymorphic serializer for objects derived from `Base`
namespace nlohmann {
    template <>
    struct adl_serializer<Base>
        : PolymorphicJsonSerializer<Base> { };
}

// Implement `Base`'s from/to functions
void to_json(json &, Base const &) { /* ... */ }
void from_json(json const &, Base &) { /* ... */ }


// Later, implement `Obj`'s from/to functions
void to_json(json &, Obj const &) { /* ... */ }
void from_json(json const &, Obj &) { /* ... */ }

// Before any serializing/deserializing of objects derived from `Base`, call the registering function for all known types.
PolymorphicJsonSerializer<Base>::register_types<Base, Obj>();

// Works!
json j = v;

Caveats:

• typeid(o).name() is unique in practice, but is not guaranteed to be by the standard. If this is an issue, it can be replaced with any persistent runtime type identification method.

• Error handling has been left out, though _serializers.at() will throw std::out_of_range when trying to serialize an unknown type.

• This implementation requires that the Base type implements its serialization with ADL from/to functions, since it takes over nlohmann::adl_serializer<Base>.

See it live on Wandbox