How can I implement a variable length row-based storage in C++?

Question

I am trying to implement a variable-length row-based storage system in C++. The idea is that each row in this storage system should be able to hold data of varying lengths and different types. For example, the first row could be [1, 8.6, "hello"], and the second row could be [2, 9.6].

To achieve this, I initially planned to use tuples with variadic templates for each row. Each row would be stored as std::tuple<Types...>, and the entire storage would be represented as a std::vector<std::tuple<Types...>>.

However, the issue I'm facing is that when I try to declare a Map class to hold this storage, I need to explicitly declare its parameter types—e.g., Map<int, double, std::string>. This approach restricts me from inserting records of different lengths and types into the same Map instance.

Here's a simplified snippet to demonstrate my point:

template <typename... Types>
class Tuple {
private:
    std::tuple<Types...> data;
public:
    Tuple() : data() {}

    Tuple(Types &&... args) : data(std::forward<Types>(args)...) {}
}

template <typename... Types>
class Map {
private:
    vector<Tuple<Types...>> record;
public:
    void addRow(Types &&... args) {
        record.emplace_back(std::forward<Types>(args)...);
    }
};

Is there a way to create a Map class where each row can hold different lengths and types of data? How can I implement this in C++?

Answer 1

You can't describe such structure, using C++ syntax. Type definitions in C++ are setting relevant object's storage representation in stone, once program is compiled.

That doesn't mean that this is isn't possible. You have two options:

1. Create a dynamic structure by using polymorphism. See Adapter, Proxy, Flyweight patterns.

2. If you REALLY need a contiguous structure (stream) of data in memory, you're on a long and thorny path to create own memory pool or protobuf library. can you use one of those existing ones?