How to load all the data from a SQL database to Hazelcast cloud?

I am using Hazelcast cloud as a caching layer for my .net application, and i want to initially load all the data from the database onto the cache. Right now, I'm accomplishing this by adding a row at a time to the cache as shown below.

 public async Task<bool> InitCache()
    {
        IEnumerable<UserV2> users = _context.UserV2s.ToList();
        var absExpTime = TimeSpan.FromHours(24);
        foreach (UserV2 user in users)
        {
            await SetRecordAsync<UserV2>(user.UserName, user, absExpTime);
        }
        return true;
    }

Is there a mechanism that allows the caching layer to directly listen to the database and fetch the data from there?

Solution

There are two recommended tools for data population in Hazelcast: MapLoader and Pipelines. Both are server-side services to load data to the Hazelcast cluster. Both require some Java coding.

MapLoader is available in all recent Hazelcast versions. It requires more code as it's a lower-level API. You must write the complete code to load the data, server executes it when applicable.

Pipelines are higher level, composable API with OOTB connectors for many databases. There is also a CDC connector for some databases that listens for database changes and updates the cache with it. Pipelines provide higher degree of fault tolerance, management and performance. It is, however, available just in Hazelcast 5.0 and beyond. With older Hazelcast, you have to use another cluster with Hazelcast Jet engine that loads the data and terminates.

See the comparison here: https://docs.hazelcast.com/hazelcast/5.0-beta-1/ingest/overview.html

For purely .NET solution, you have to manage the data loading yourself using the .NET client and a custom data loader.